Modeling text may not be as obvious as modeling images or audio.
In images, inputs are a collection of pixels. In audio, it can be the vector obtained from a spectrogram (i.e. Fourier transform through time). Both representations share a common feature. Both are dense.
When we consider the problem of natural language processing (NLP), things differ a little.
Let us first check on the most simple representation of language. In particular, document representation.
In text classification for example, we are given a description $x \in {\bf R}^d$ of a document $\delta$ and a fixed set of classes $y \in \{c_1, \dots, c_K\}$, for example the document topic. Given a new document, our goal is to predict the most probable class.
A very simple description of a document is the bag-of-words description. This representation transforms a complete text to a vector of $d$ predefined words. The set of predefined words is selected by the practicioner. For example, the list can consist of the set of all words in a given language.
Example 1: Suppose we are given four different documents belonging to the topics $y=\{\text{'economics'},\text{'technology'}\}$ and we select as our representation the following bag-of-words $x = \{\text{'market'}, \text{'stock'}, \text{'price'}, \text{'application'}, \text{'mobile'}, \text{'google'}\}$. We can count the number of times a certain term appears in that document and expect that this description is discriminative enough for identifying the document topic. Check the following example:
| market | stock | price | application | mobile | ||
| document 1($\text{'economics'}$) | 1 | 2 | 3 | 0 | 0 | 0 |
| document 2($\text{'economics'}$) | 0 | 1 | 2 | 0 | 0 | 1 |
| document 3($\text{'technology'}$) | 0 | 0 | 0 | 2 | 3 | 1 |
| document 4($\text{'technology'}$) | 1 | 0 | 1 | 2 | 3 | 0 |
In this representation, document 2 is represented by the vector (0,1,2,0,0,1). We can alternatively use a binary value representing whether a term appears or not in the document. In this last case document would be represesnted by (0,1,1,0,0,1).
Observe that this is a context free representation, i.e. the order of the words is not considered. Consider the sentences "Google reduces the prices of applications in App market" and "The number of aplications in Google App market with cheap prices is reduced by 20%". The representation for both sentences is the same, though the exact meaning of both sentences is completely different. However, this kind of representation may be enough for identifying that both refers to $\text{'technology'}$.
A different approach for working with NLP is considering the embedding of single words, i.e. looking for a manifold where semantically similar words are mapped to nearby points. These are called vector space models.
The term word embedding was introduced by Bengio et al. at the begining of the 2000's. However it was Mikolov et al. in 2013 with the creation of word2vec who popularized word embeddings. From then, different word embeddings using deep learning appeared. It is worth mentioning GloVe from Pennington et al. (2014).
The term embedding naturally appears in any deep architecture where there is a bottleneck layer. The output of that bottleneck layer can be seen as low-dimensional embedding. This idea lies at the core of deep learning, and we have seen that before in the unsupervised notebook.
In this sense we can easily distinguish among task-oriented embeddings and general purpose embeddings:
Task oriented embeddings has the Embedding layer a one of the layers for a different task. This is, while solving the problem at hand, one also learns a suitable embedding for that task.
General purpose embeddings are designed to be used or transferred across different tasks.
Contrary to other embeddings, such as the ones we found in the image domain, the word embedding layer usually regarded as a mapping from a discrete set of objects (words) to a real valued vector, i.e.
$$k\in\{1..N\} \rightarrow \mathbb{R}^d$$Thus, we can represent the Embedding layer as $N\times d$ matrix, or just a table/dictionary.
$$ \begin{matrix} word1: \\ word2:\\ \vdots\\ wordN: \\ \end{matrix} \left[ \begin{matrix} x_{1,1}&x_{1,2}& \dots &x_{1,d}\\ x_{2,1}&x_{2,2}& \dots &x_{2,d}\\ \vdots&&\\ x_{N,1}&x_{N,2}& \dots &x_{N,d}\\ \end{matrix} \right] $$In this sense, the basic operation that an embedding layer has to accomplish is that given a certain word it returns the assigned code. And the goal in learning is to learn the values in the matrix.
In the learning process the matrix is initialized at random and learned using standard learning procedures, such as backpropagation.
In this section, we are going to check two of the most well known strategies for learning general purpose embeddings, namely CBOW and Skip-gram.
The idea behind both methods is simple, the context around a word is a hint with respect the underlying semantic of that word, i.e. if we find the same context around the target words in different sentences but those words are different, most probably they convey the same meaning.
Strictly following this idea, we can define CBOW (Continuous Bag-of-Words): Given the context of a word, i.e. the $k$ words around the target, we want to infer what the target word is.
We can however change this order, and this will gives us the Skip-gram. In the skip-gram, given one word, we want to predict its context.
In [1]:
%matplotlib inline
from __future__ import print_function
import collections
import math
import numpy as np
import os
import random
import tensorflow as tf
import zipfile
from matplotlib import pylab
from six.moves import range
from six.moves.urllib.request import urlretrieve
from sklearn.manifold import TSNE
/usr/local/lib/python2.7/dist-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
In [12]:
url = 'http://mattmahoney.net/dc/'
def maybe_download(filename, expected_bytes):
"""Download a file if not present, and make sure it's the right size."""
if not os.path.exists(filename):
filename, _ = urlretrieve(url + filename, filename)
statinfo = os.stat(filename)
if statinfo.st_size == expected_bytes:
print('Found and verified %s' % filename)
else:
print(statinfo.st_size)
raise Exception(
'Failed to verify ' + filename + '. Can you get to it with a browser?')
return filename
filename = maybe_download('text8.zip', 31344016)
Found and verified text8.zip
In [3]:
def read_data(filename):
"""Extract the first file enclosed in a zip file as a list of words"""
with zipfile.ZipFile(filename) as f:
data = tf.compat.as_str(f.read(f.namelist()[0])).split()
return data
words = read_data(filename)
print('Data size %d' % len(words))
Data size 17005207
In [4]:
vocabulary_size = 50000
def build_dataset(words):
count = [['UNK', -1]]
count.extend(collections.Counter(words).most_common(vocabulary_size - 1))
dictionary = dict()
for word, _ in count:
dictionary[word] = len(dictionary)
data = list()
unk_count = 0
for word in words:
if word in dictionary:
index = dictionary[word]
else:
index = 0 # dictionary['UNK']
unk_count = unk_count + 1
data.append(index)
count[0][1] = unk_count
reverse_dictionary = dict(zip(dictionary.values(), dictionary.keys()))
return data, count, dictionary, reverse_dictionary
data, count, dictionary, reverse_dictionary = build_dataset(words)
print('Most common words (+UNK)', count[:5])
print('Sample data', data[:10])
del words # Hint to reduce memory.
Most common words (+UNK) [['UNK', 418391], ('the', 1061396), ('of', 593677), ('and', 416629), ('one', 411764)]
Sample data [5239, 3084, 12, 6, 195, 2, 3137, 46, 59, 156]
In [5]:
data_index = 0
def generate_batch(batch_size, num_skips, skip_window):
global data_index
assert batch_size % num_skips == 0
assert num_skips <= 2 * skip_window
batch = np.ndarray(shape=(batch_size), dtype=np.int32)
labels = np.ndarray(shape=(batch_size, 1), dtype=np.int32)
span = 2 * skip_window + 1 # [ skip_window target skip_window ]
buffer = collections.deque(maxlen=span)
for _ in range(span):
buffer.append(data[data_index])
data_index = (data_index + 1) % len(data)
for i in range(batch_size // num_skips):
target = skip_window # target label at the center of the buffer
targets_to_avoid = [ skip_window ]
for j in range(num_skips):
while target in targets_to_avoid:
target = random.randint(0, span - 1)
targets_to_avoid.append(target)
batch[i * num_skips + j] = buffer[skip_window]
labels[i * num_skips + j, 0] = buffer[target]
buffer.append(data[data_index])
data_index = (data_index + 1) % len(data)
return batch, labels
print('data:', [reverse_dictionary[di] for di in data[:8]])
for num_skips, skip_window in [(2, 1), (4, 2)]:
data_index = 0
batch, labels = generate_batch(batch_size=8, num_skips=num_skips, skip_window=skip_window)
print('\nwith num_skips = %d and skip_window = %d:' % (num_skips, skip_window))
print(' batch:', [reverse_dictionary[bi] for bi in batch])
print(' labels:', [reverse_dictionary[li] for li in labels.reshape(8)])
data: ['anarchism', 'originated', 'as', 'a', 'term', 'of', 'abuse', 'first']
with num_skips = 2 and skip_window = 1:
batch: ['originated', 'originated', 'as', 'as', 'a', 'a', 'term', 'term']
labels: ['as', 'anarchism', 'originated', 'a', 'term', 'as', 'a', 'of']
with num_skips = 4 and skip_window = 2:
batch: ['as', 'as', 'as', 'as', 'a', 'a', 'a', 'a']
labels: ['originated', 'a', 'term', 'anarchism', 'originated', 'term', 'as', 'of']
In [6]:
batch_size = 128
embedding_size = 128 # Dimension of the embedding vector.
skip_window = 1 # How many words to consider left and right.
num_skips = 2 # How many times to reuse an input to generate a label.
# We pick a random validation set to sample nearest neighbors. here we limit the
# validation samples to the words that have a low numeric ID, which by
# construction are also the most frequent.
valid_size = 16 # Random set of words to evaluate similarity on.
valid_window = 100 # Only pick dev samples in the head of the distribution.
valid_examples = np.array(random.sample(range(valid_window), valid_size))
num_sampled = 64 # Number of negative examples to sample.
graph = tf.Graph()
with graph.as_default(), tf.device('/cpu:0'):
# Input data.
train_dataset = tf.placeholder(tf.int32, shape=[batch_size])
train_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])
valid_dataset = tf.constant(valid_examples, dtype=tf.int32)
# Variables.
embeddings = tf.Variable(tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))
softmax_weights = tf.Variable(tf.truncated_normal([vocabulary_size, embedding_size],stddev=1.0 / math.sqrt(embedding_size)))
softmax_biases = tf.Variable(tf.zeros([vocabulary_size]))
# Model.
# Look up embeddings for inputs.
embed = tf.nn.embedding_lookup(embeddings, train_dataset)
# Compute the softmax loss, using a sample of the negative labels each time.
loss = tf.reduce_mean(
tf.nn.sampled_softmax_loss(softmax_weights, softmax_biases, embed,
train_labels, num_sampled, vocabulary_size))
# Optimizer.
# Note: The optimizer will optimize the softmax_weights AND the embeddings.
# This is because the embeddings are defined as a variable quantity and the
# optimizer's `minimize` method will by default modify all variable quantities
# that contribute to the tensor it is passed.
# See docs on `tf.train.Optimizer.minimize()` for more details.
optimizer = tf.train.AdagradOptimizer(1.0).minimize(loss)
# Compute the similarity between minibatch examples and all embeddings.
# We use the cosine distance:
norm = tf.sqrt(tf.reduce_sum(tf.square(embeddings), 1, keep_dims=True))
normalized_embeddings = embeddings / norm
valid_embeddings = tf.nn.embedding_lookup(
normalized_embeddings, valid_dataset)
similarity = tf.matmul(valid_embeddings, tf.transpose(normalized_embeddings))
In [7]:
num_steps = 1000001
with tf.Session(graph=graph) as session:
tf.initialize_all_variables().run()
print('Initialized')
average_loss = 0
for step in range(num_steps):
batch_data, batch_labels = generate_batch(
batch_size, num_skips, skip_window)
feed_dict = {train_dataset : batch_data, train_labels : batch_labels}
_, l = session.run([optimizer, loss], feed_dict=feed_dict)
average_loss += l
if step % 2000 == 0:
if step > 0:
average_loss = average_loss / 2000
# The average loss is an estimate of the loss over the last 2000 batches.
print('Average loss at step %d: %f' % (step, average_loss))
average_loss = 0
# note that this is expensive (~20% slowdown if computed every 500 steps)
if step % 10000 == 0:
sim = similarity.eval()
for i in range(valid_size):
valid_word = reverse_dictionary[valid_examples[i]]
top_k = 8 # number of nearest neighbors
nearest = (-sim[i, :]).argsort()[1:top_k+1]
log = 'Nearest to %s:' % valid_word
for k in range(top_k):
close_word = reverse_dictionary[nearest[k]]
log = '%s %s,' % (log, close_word)
print(log)
final_embeddings = normalized_embeddings.eval()
Initialized
Average loss at step 0: 8.043100
Nearest to this: reichstag, chapelle, ruskin, clog, chronos, knud, poorer, wickets,
Nearest to more: libra, carinthia, unpaired, gunman, linebacker, spaulding, landscaping, detonate,
Nearest to system: storks, mau, democratically, owing, reeve, lead, calculations, cents,
Nearest to his: irritability, morin, publically, dowling, munster, stabbing, cjd, thoroughbreds,
Nearest to while: musab, samba, jamming, revolution, winch, tourist, assault, brewery,
Nearest to after: inserting, chrysotile, umno, unconsciously, vaccination, suing, streamline, goldoni,
Nearest to by: contingency, mudslides, notch, tears, operation, iqaluit, liberalize, platt,
Nearest to has: medium, braudel, cutting, tabasco, abilities, fabulous, moi, camelids,
Nearest to six: plumbing, dominance, participants, valentine, dredd, szlachta, branagh, latitude,
Nearest to seven: hortense, vibration, loach, maiden, furnish, clinching, examined, dhaka,
Nearest to only: herds, jury, dissertation, programming, biking, wedgwood, runways, myanmar,
Nearest to were: tennis, prohibits, flowering, coloration, erin, stomping, esau, infringing,
Nearest to two: scarborough, mohel, lamanites, enlistment, vines, degeneration, deficit, merged,
Nearest to for: emacs, geeks, contributing, print, combatants, businessmen, sober, derry,
Nearest to at: tremors, skiego, wg, unprovoked, czechoslovak, gelling, eml, nesting,
Nearest to history: folding, reliever, trusteeship, scotia, mothersbaugh, joaqu, witten, walker,
Average loss at step 2000: 4.363188
Average loss at step 4000: 3.862560
Average loss at step 6000: 3.792075
Average loss at step 8000: 3.690573
Average loss at step 10000: 3.620293
Nearest to this: it, which, knud, the, sidewalks, that, nokia, some,
Nearest to more: gunman, immortals, libra, tackle, dismounted, bn, carinthia, gauze,
Nearest to system: climbed, detroit, democratically, reeve, mau, owing, tocantins, substandard,
Nearest to his: their, her, the, s, hvac, its, imitation, a,
Nearest to while: musab, samba, revolution, tourist, winch, sinus, personal, numeral,
Nearest to after: before, notre, between, kaj, chrysotile, cocktails, within, leo,
Nearest to by: was, in, with, against, be, attacking, geographical, supremely,
Nearest to has: had, was, is, have, aegis, phenotype, tabasco, became,
Nearest to six: eight, four, seven, five, three, nine, zero, two,
Nearest to seven: six, eight, four, five, nine, three, zero, two,
Nearest to only: pyramids, jury, dimorphism, ram, uncover, parish, pusan, pervasive,
Nearest to were: are, was, have, rachel, had, elbow, dcs, bela,
Nearest to two: three, four, five, six, seven, eight, zero, one,
Nearest to for: in, with, hiker, overture, of, boundary, mclachlan, kamov,
Nearest to at: in, dye, czechoslovak, from, tremors, on, teflon, commentators,
Nearest to history: reliever, folding, scotia, reviewing, joaqu, everyone, rebuffed, ikea,
Average loss at step 12000: 3.609490
Average loss at step 14000: 3.566351
Average loss at step 16000: 3.412137
Average loss at step 18000: 3.460783
Average loss at step 20000: 3.542671
Nearest to this: it, which, pfaff, his, sidewalks, some, any, knud,
Nearest to more: less, most, describing, dismounted, tackle, bn, hooke, accrue,
Nearest to system: detroit, ocogs, reeve, climbed, democratically, owing, intellivision, gallbladder,
Nearest to his: their, her, its, the, strat, this, relegated, teeming,
Nearest to while: samba, musab, navel, zenith, winch, tourist, facilitator, revolution,
Nearest to after: before, within, between, kaj, chrysotile, notre, during, later,
Nearest to by: for, be, against, between, attacking, piloted, croatian, in,
Nearest to has: had, was, is, have, longer, rich, aegis, erythrocytes,
Nearest to six: five, eight, three, four, seven, nine, zero, two,
Nearest to seven: eight, four, five, six, three, nine, zero, two,
Nearest to only: ram, pyramids, jury, lepers, retarded, pervasive, dimorphism, you,
Nearest to were: are, was, had, have, be, is, been, dcs,
Nearest to two: three, five, six, four, eight, seven, one, zero,
Nearest to for: by, in, from, of, reassure, valerian, clu, calling,
Nearest to at: in, cooke, from, lamentations, on, terran, with, is,
Nearest to history: reliever, journalism, rebuffed, reviewing, familias, list, oscar, jody,
Average loss at step 22000: 3.504021
Average loss at step 24000: 3.485173
Average loss at step 26000: 3.482792
Average loss at step 28000: 3.479294
Average loss at step 30000: 3.504650
Nearest to this: it, which, what, another, that, some, knud, pfaff,
Nearest to more: less, rather, most, very, hsieh, sorority, tackle, greater,
Nearest to system: systems, smiths, inappropriately, uzziah, owing, outlying, reeve, fanatics,
Nearest to his: their, her, its, s, the, strat, neale, munster,
Nearest to while: navel, when, though, samba, zenith, although, where, winch,
Nearest to after: before, during, within, butyl, when, despite, for, kaj,
Nearest to by: geographical, against, in, with, exacted, been, were, was,
Nearest to has: had, have, is, was, longer, having, camelids, zoroastrianism,
Nearest to six: eight, seven, four, five, nine, three, zero, two,
Nearest to seven: eight, six, nine, four, five, three, zero, two,
Nearest to only: jury, retarded, ram, pyramids, bribed, accredited, lightweight, match,
Nearest to were: are, was, have, had, be, is, been, eroded,
Nearest to two: four, three, one, seven, five, six, eight, nine,
Nearest to for: of, hiker, with, unassigned, against, after, when, hogeschool,
Nearest to at: in, goblin, when, czechoslovak, dye, on, during, symbolizing,
Nearest to history: journalism, list, reliever, outside, lustre, declination, familias, magadha,
Average loss at step 32000: 3.504751
Average loss at step 34000: 3.493520
Average loss at step 36000: 3.454435
Average loss at step 38000: 3.303658
Average loss at step 40000: 3.429776
Nearest to this: which, it, that, any, the, what, another, pfaff,
Nearest to more: less, most, very, greater, hsieh, rather, souls, observational,
Nearest to system: systems, intellivision, owing, facilities, services, uzziah, smiths, sofia,
Nearest to his: their, her, its, s, strat, lameness, my, the,
Nearest to while: when, though, where, although, but, was, navel, myles,
Nearest to after: before, during, when, fifty, backpacker, armin, within, without,
Nearest to by: sul, exacted, supranationalism, croatian, otranto, be, when, for,
Nearest to has: had, have, was, is, replacing, since, hijackers, fairer,
Nearest to six: five, seven, eight, four, nine, three, two, one,
Nearest to seven: eight, five, six, nine, three, four, zero, one,
Nearest to only: it, gunshot, approximately, jury, stalling, no, retarded, just,
Nearest to were: are, was, have, be, had, being, became, been,
Nearest to two: three, four, five, six, seven, one, eight, zero,
Nearest to for: photographic, from, when, businessmen, by, overture, unassigned, hiker,
Nearest to at: on, during, czechoslovak, in, dye, lamentations, terran, goblin,
Nearest to history: list, journalism, declination, reign, collaboration, heritage, hazards, life,
Average loss at step 42000: 3.437502
Average loss at step 44000: 3.453602
Average loss at step 46000: 3.449825
Average loss at step 48000: 3.356782
Average loss at step 50000: 3.381733
Nearest to this: which, it, the, that, he, wilmut, chalmers, another,
Nearest to more: less, most, very, greater, rather, higher, deeds, hsieh,
Nearest to system: systems, owing, uzziah, facilities, lightsabers, services, texaco, colloidal,
Nearest to his: their, her, its, my, strat, our, him, your,
Nearest to while: when, although, where, though, but, after, however, did,
Nearest to after: before, during, when, while, despite, within, but, afc,
Nearest to by: was, exacted, through, with, in, under, piloted, sul,
Nearest to has: had, have, is, was, having, since, contains, fairer,
Nearest to six: eight, seven, four, five, nine, three, zero, one,
Nearest to seven: six, eight, nine, four, five, three, zero, two,
Nearest to only: stalling, jury, slice, first, argentines, usually, still, omaha,
Nearest to were: are, have, was, had, be, been, those, niches,
Nearest to two: three, four, six, one, five, seven, eight, zero,
Nearest to for: jingles, while, hiker, photographic, mosquitia, including, ldp, among,
Nearest to at: during, ducati, lamentations, in, on, braun, under, from,
Nearest to history: list, journalism, crown, period, encyclopedia, baseball, sanger, foe,
Average loss at step 52000: 3.436932
Average loss at step 54000: 3.426540
Average loss at step 56000: 3.434931
Average loss at step 58000: 3.396788
Average loss at step 60000: 3.400008
Nearest to this: which, it, the, that, what, there, itself, some,
Nearest to more: less, most, very, rather, greater, better, higher, larger,
Nearest to system: systems, owing, facilities, services, smiths, computer, flickr, rosicrucians,
Nearest to his: their, her, its, my, your, strat, our, the,
Nearest to while: when, although, though, after, were, before, but, where,
Nearest to after: before, during, despite, while, when, within, until, from,
Nearest to by: through, supranationalism, with, orthogonality, mantis, piloted, sulfate, bacon,
Nearest to has: had, have, was, having, is, contains, showed, willful,
Nearest to six: eight, nine, four, five, seven, three, zero, two,
Nearest to seven: eight, six, five, four, nine, three, zero, two,
Nearest to only: first, jury, usually, platonist, stalling, still, accredited, omaha,
Nearest to were: are, was, have, while, although, those, had, including,
Nearest to two: four, three, five, six, one, seven, eight, zero,
Nearest to for: of, without, kamov, mclachlan, detecting, after, hiker, calling,
Nearest to at: ducati, during, in, lamentations, czechoslovak, lombardo, lightweight, fets,
Nearest to history: list, politics, demographics, heritage, period, europe, hazards, designers,
Average loss at step 62000: 3.239837
Average loss at step 64000: 3.253091
Average loss at step 66000: 3.402658
Average loss at step 68000: 3.391684
Average loss at step 70000: 3.360692
Nearest to this: which, it, another, the, that, what, audita, some,
Nearest to more: less, most, very, greater, better, higher, rather, increasingly,
Nearest to system: systems, flatted, program, owing, scheme, excelling, consort, services,
Nearest to his: her, their, its, my, our, your, affirms, payment,
Nearest to while: although, though, when, where, before, and, but, however,
Nearest to after: before, during, despite, while, when, until, without, backpacker,
Nearest to by: through, supranationalism, be, exacted, is, in, mantis, orthogonality,
Nearest to has: had, have, is, was, contains, since, having, attenuation,
Nearest to six: seven, eight, four, five, nine, three, zero, two,
Nearest to seven: six, eight, four, nine, five, three, zero, one,
Nearest to only: jury, retarded, still, directly, bilabial, patented, strictly, platonist,
Nearest to were: are, was, have, be, had, include, including, while,
Nearest to two: three, four, six, seven, five, one, eight, zero,
Nearest to for: without, to, including, in, detonator, unassigned, against, hiker,
Nearest to at: headlined, on, lamentations, during, czechoslovak, armored, goblin, versus,
Nearest to history: list, consists, encyclopedia, heritage, designers, collection, scorpions, museum,
Average loss at step 72000: 3.371133
Average loss at step 74000: 3.348847
Average loss at step 76000: 3.313980
Average loss at step 78000: 3.349819
Average loss at step 80000: 3.377335
Nearest to this: which, it, the, another, that, what, there, knud,
Nearest to more: less, most, very, better, older, garret, rather, higher,
Nearest to system: systems, owing, lightsabers, program, flatted, uzziah, scheme, intellivision,
Nearest to his: her, their, its, my, your, the, strat, our,
Nearest to while: although, though, when, however, after, but, including, or,
Nearest to after: before, during, when, despite, while, until, without, through,
Nearest to by: through, against, supranationalism, williams, exacted, was, sul, prostitution,
Nearest to has: had, have, is, was, having, since, contains, but,
Nearest to six: seven, five, four, eight, three, nine, two, zero,
Nearest to seven: six, five, eight, nine, four, three, zero, one,
Nearest to only: jury, astrophysical, impairments, directly, approximately, compromised, accredited, seleucid,
Nearest to were: are, was, have, had, became, being, those, include,
Nearest to two: three, four, six, five, seven, one, eight, zero,
Nearest to for: ldp, hiker, including, parental, trouv, foremost, in, without,
Nearest to at: in, on, during, while, armored, versus, goblin, headlined,
Nearest to history: list, encyclopedia, designers, magadha, foe, development, halfway, collection,
Average loss at step 82000: 3.406093
Average loss at step 84000: 3.407716
Average loss at step 86000: 3.389837
Average loss at step 88000: 3.354962
Average loss at step 90000: 3.365933
Nearest to this: which, it, some, another, the, similar, conjugated, cation,
Nearest to more: less, very, most, better, garret, increasingly, greater, quite,
Nearest to system: systems, program, flatted, owing, metropolis, lightsabers, canonic, abruzzo,
Nearest to his: her, their, its, your, my, our, the, s,
Nearest to while: although, though, when, after, before, during, however, but,
Nearest to after: before, during, despite, while, until, when, without, for,
Nearest to by: through, without, under, supranationalism, exacted, when, against, sul,
Nearest to has: had, have, is, having, was, since, makes, contains,
Nearest to six: eight, seven, four, five, nine, three, zero, two,
Nearest to seven: eight, six, nine, five, four, three, zero, one,
Nearest to only: jury, approximately, no, neither, just, astrophysical, match, compromised,
Nearest to were: are, was, had, have, became, while, these, is,
Nearest to two: three, four, five, six, seven, eight, one, zero,
Nearest to for: including, of, after, without, against, trouv, unassigned, while,
Nearest to at: during, on, goblin, under, headlined, in, near, lombardo,
Nearest to history: list, heritage, magadha, politics, reductionist, designers, rico, philosophy,
Average loss at step 92000: 3.397715
Average loss at step 94000: 3.256335
Average loss at step 96000: 3.359186
Average loss at step 98000: 3.243878
Average loss at step 100000: 3.357087
Nearest to this: it, which, another, itself, the, culminates, resigning, kites,
Nearest to more: less, most, very, better, garret, greater, quite, increasingly,
Nearest to system: systems, program, canonic, lightsabers, flatted, platform, excelling, demanding,
Nearest to his: her, their, your, my, its, the, our, s,
Nearest to while: although, when, though, before, but, where, however, after,
Nearest to after: before, during, despite, when, without, until, while, through,
Nearest to by: without, against, through, exacted, under, been, when, piloted,
Nearest to has: had, have, is, was, having, since, attenuation, bord,
Nearest to six: seven, four, eight, five, nine, three, zero, two,
Nearest to seven: eight, four, nine, six, five, three, zero, two,
Nearest to only: jury, approximately, no, never, still, not, just, reciprocated,
Nearest to were: are, have, these, had, was, those, include, including,
Nearest to two: four, three, five, six, seven, eight, one, zero,
Nearest to for: without, including, after, at, mclachlan, before, against, hiker,
Nearest to at: on, in, during, within, for, headlined, from, lombardo,
Nearest to history: list, heritage, reductionist, mencken, magadha, petition, encyclopedia, designers,
Average loss at step 102000: 3.337031
Average loss at step 104000: 3.318722
Average loss at step 106000: 3.352940
Average loss at step 108000: 3.340055
Average loss at step 110000: 3.228543
Nearest to this: which, it, each, some, the, what, that, another,
Nearest to more: less, very, most, rather, increasingly, better, quite, older,
Nearest to system: systems, program, services, facilities, flatted, anatomy, abruzzo, latvijas,
Nearest to his: her, their, its, my, your, our, strat, conant,
Nearest to while: although, though, when, after, or, where, before, however,
Nearest to after: before, during, despite, while, when, without, until, although,
Nearest to by: through, in, against, was, exacted, under, and, is,
Nearest to has: had, have, is, was, having, since, bord, attenuation,
Nearest to six: seven, four, eight, five, three, nine, zero, two,
Nearest to seven: six, eight, four, five, nine, three, zero, two,
Nearest to only: approximately, jury, until, platonist, just, achaemenid, flourishes, exec,
Nearest to were: are, was, have, had, these, those, is, became,
Nearest to two: three, one, four, seven, six, five, eight, zero,
Nearest to for: and, of, without, in, after, climatic, by, including,
Nearest to at: during, in, near, goblin, furigana, lombardo, within, under,
Nearest to history: list, philosophy, heritage, encyclopedia, politics, period, europe, continuation,
Average loss at step 112000: 3.263437
Average loss at step 114000: 3.315625
Average loss at step 116000: 3.316637
Average loss at step 118000: 3.317571
Average loss at step 120000: 3.310561
Nearest to this: it, which, what, any, similar, another, itself, each,
Nearest to more: less, most, very, older, rather, better, quite, increasingly,
Nearest to system: systems, program, flatted, latvijas, rule, revenue, excelling, convey,
Nearest to his: her, their, its, your, my, the, our, whose,
Nearest to while: although, though, when, but, where, however, after, before,
Nearest to after: before, during, despite, when, while, until, without, for,
Nearest to by: through, during, against, under, with, from, was, supranationalism,
Nearest to has: had, have, is, having, since, was, attenuation, contains,
Nearest to six: four, seven, eight, five, nine, three, zero, two,
Nearest to seven: eight, five, six, nine, four, three, zero, one,
Nearest to only: approximately, jury, all, divisional, until, ortega, thrilling, vaz,
Nearest to were: are, was, have, had, became, those, be, include,
Nearest to two: three, four, six, five, seven, eight, one, nine,
Nearest to for: without, after, during, hiker, when, at, mclachlan, nkvd,
Nearest to at: in, during, under, goblin, within, near, fets, for,
Nearest to history: list, philosophy, graduates, heritage, politics, europe, title, aquilonia,
Average loss at step 122000: 3.319173
Average loss at step 124000: 3.248780
Average loss at step 126000: 3.204769
Average loss at step 128000: 3.290055
Average loss at step 130000: 3.206077
Nearest to this: it, which, another, the, each, any, wilmut, spontaneous,
Nearest to more: less, most, very, rather, better, larger, fewer, greater,
Nearest to system: systems, program, facilities, texaco, panel, latvijas, trophies, process,
Nearest to his: her, their, your, my, its, the, our, whose,
Nearest to while: although, when, though, however, including, before, or, after,
Nearest to after: before, during, despite, when, without, until, within, since,
Nearest to by: with, through, exacted, including, after, without, supranationalism, piloted,
Nearest to has: had, have, contains, is, having, since, was, makes,
Nearest to six: eight, seven, four, five, three, nine, two, zero,
Nearest to seven: six, eight, five, four, nine, three, zero, two,
Nearest to only: approximately, jury, full, until, no, grotto, trimester, vaz,
Nearest to were: are, was, have, had, include, been, those, be,
Nearest to two: three, four, six, five, seven, eight, zero, nine,
Nearest to for: without, against, parental, shura, with, paulette, adowa, unprovable,
Nearest to at: in, near, during, under, armored, genius, on, haskell,
Nearest to history: list, heritage, period, politics, science, title, cpsu, graduates,
Average loss at step 132000: 3.238015
Average loss at step 134000: 3.277674
Average loss at step 136000: 3.191930
Average loss at step 138000: 3.331734
Average loss at step 140000: 3.295731
Nearest to this: which, it, that, the, similar, itself, some, maas,
Nearest to more: less, very, most, quite, better, increasingly, rather, highly,
Nearest to system: systems, program, network, facilities, services, revenue, process, latvijas,
Nearest to his: her, their, my, your, its, whose, our, reckoning,
Nearest to while: although, when, though, however, before, after, during, but,
Nearest to after: before, during, despite, when, while, until, without, for,
Nearest to by: through, using, with, supranationalism, during, without, against, orthogonality,
Nearest to has: had, have, contains, was, is, having, since, uses,
Nearest to six: eight, five, seven, four, nine, three, zero, two,
Nearest to seven: eight, six, nine, five, four, three, zero, two,
Nearest to only: approximately, compromised, jury, until, solved, usually, astrophysical, still,
Nearest to were: are, was, have, had, those, include, be, been,
Nearest to two: three, five, four, six, zero, seven, eight, one,
Nearest to for: without, after, including, if, parental, while, intelligentsia, hiker,
Nearest to at: near, in, goblin, under, on, ducati, undecided, around,
Nearest to history: list, heritage, period, development, graduates, science, continuation, politics,
Average loss at step 142000: 3.264663
Average loss at step 144000: 3.113093
Average loss at step 146000: 3.272680
Average loss at step 148000: 3.299780
Average loss at step 150000: 3.304640
Nearest to this: which, it, another, some, instead, vdp, his, each,
Nearest to more: less, most, very, better, rather, too, quite, increasingly,
Nearest to system: systems, program, network, facilities, services, excelling, colloidal, handily,
Nearest to his: her, their, my, your, its, whose, our, the,
Nearest to while: although, though, when, however, before, during, if, but,
Nearest to after: before, during, despite, when, without, for, within, until,
Nearest to by: through, when, during, yah, with, masers, guillotined, without,
Nearest to has: had, have, was, is, contains, having, since, arezzo,
Nearest to six: eight, seven, five, four, nine, three, zero, two,
Nearest to seven: eight, six, five, four, nine, three, zero, two,
Nearest to only: approximately, jury, compromised, best, astrophysical, platonist, latter, cmyk,
Nearest to were: are, was, have, had, is, include, those, been,
Nearest to two: three, five, four, six, seven, eight, one, zero,
Nearest to for: without, after, before, including, when, hiker, mclachlan, during,
Nearest to at: in, near, under, fets, goblin, unlucky, on, haskell,
Nearest to history: list, science, period, heritage, tradition, politics, philosophy, graduates,
Average loss at step 152000: 3.323575
Average loss at step 154000: 3.312371
Average loss at step 156000: 3.282100
Average loss at step 158000: 3.153083
Average loss at step 160000: 3.269632
Nearest to this: which, another, it, that, similar, itself, deterrence, vdp,
Nearest to more: less, most, better, rather, very, greater, quite, highly,
Nearest to system: systems, program, reform, facilities, services, flatted, district, trophies,
Nearest to his: her, their, its, my, your, whose, the, our,
Nearest to while: although, though, when, however, before, but, where, after,
Nearest to after: before, during, despite, when, while, without, since, though,
Nearest to by: against, including, through, during, with, when, from, without,
Nearest to has: had, have, is, having, contains, includes, makes, was,
Nearest to six: five, eight, seven, four, nine, three, zero, two,
Nearest to seven: eight, six, five, nine, four, three, zero, two,
Nearest to only: jury, approximately, generally, universally, strictly, neither, compromised, actually,
Nearest to were: are, was, those, these, have, been, include, being,
Nearest to two: three, five, four, seven, eight, six, one, zero,
Nearest to for: without, including, against, despite, while, jingles, after, mclachlan,
Nearest to at: in, near, during, grammy, under, fets, lombardo, goblin,
Nearest to history: list, period, tradition, philosophy, heritage, science, dictionary, reign,
Average loss at step 162000: 3.325408
Average loss at step 164000: 3.249799
Average loss at step 166000: 3.236623
Average loss at step 168000: 3.276600
Average loss at step 170000: 3.270730
Nearest to this: which, it, another, each, deterrence, similar, audita, the,
Nearest to more: less, very, most, better, rather, quite, highly, greater,
Nearest to system: systems, program, scheme, facilities, unit, practice, trophies, network,
Nearest to his: her, their, my, its, your, whose, our, the,
Nearest to while: although, though, when, before, but, after, however, if,
Nearest to after: before, during, despite, following, while, when, without, since,
Nearest to by: through, from, under, against, with, piloted, newly, for,
Nearest to has: had, have, is, having, was, contains, bord, holds,
Nearest to six: seven, eight, five, nine, four, three, two, zero,
Nearest to seven: eight, six, five, four, nine, three, zero, two,
Nearest to only: approximately, match, no, sen, cameo, jury, mourns, adversely,
Nearest to were: are, have, was, had, those, include, including, been,
Nearest to two: three, four, six, five, seven, eight, one, nine,
Nearest to for: after, without, despite, against, parental, ldp, by, mclachlan,
Nearest to at: near, in, during, on, goblin, fets, lombardo, dye,
Nearest to history: list, scorpions, northern, cpsu, western, see, montrose, india,
Average loss at step 172000: 3.296613
Average loss at step 174000: 3.285630
Average loss at step 176000: 3.246882
Average loss at step 178000: 3.141528
Average loss at step 180000: 3.272591
Nearest to this: which, it, another, the, some, there, each, what,
Nearest to more: less, most, very, better, rather, increasingly, greater, quite,
Nearest to system: systems, network, trophies, boulevards, flatted, program, platform, scheme,
Nearest to his: her, their, your, its, my, our, whose, the,
Nearest to while: although, though, when, however, after, before, but, despite,
Nearest to after: before, during, without, despite, when, while, since, within,
Nearest to by: through, without, when, exacted, against, supranationalism, was, scarcely,
Nearest to has: had, have, is, contains, was, uses, provides, requires,
Nearest to six: seven, four, five, eight, nine, three, zero, two,
Nearest to seven: six, eight, five, nine, four, three, zero, one,
Nearest to only: approximately, usually, eagerly, no, equally, strictly, until, grotto,
Nearest to were: are, was, had, have, these, those, including, while,
Nearest to two: three, one, four, six, five, seven, zero, eight,
Nearest to for: without, of, if, after, against, parental, unassigned, despite,
Nearest to at: near, in, during, goblin, fets, lombardo, furigana, dye,
Nearest to history: list, scorpions, tradition, encyclopedia, period, dictionary, incantations, see,
Average loss at step 182000: 3.208810
Average loss at step 184000: 3.259454
Average loss at step 186000: 3.256721
Average loss at step 188000: 3.237530
Average loss at step 190000: 3.153244
Nearest to this: which, it, another, physicalism, each, the, adventists, due,
Nearest to more: less, most, very, better, greater, rather, increasingly, quite,
Nearest to system: systems, program, scheme, method, reform, tarr, trophies, unit,
Nearest to his: her, their, its, your, my, our, whose, the,
Nearest to while: although, though, when, however, before, began, after, if,
Nearest to after: before, during, despite, without, when, following, while, since,
Nearest to by: supranationalism, including, through, when, honshu, walther, contingency, from,
Nearest to has: had, have, is, was, since, having, contains, attenuation,
Nearest to six: eight, seven, four, five, nine, three, zero, two,
Nearest to seven: four, five, eight, six, nine, three, zero, two,
Nearest to only: platonist, approximately, usually, mourns, eagerly, neither, until, strictly,
Nearest to were: are, was, had, have, these, be, remain, while,
Nearest to two: three, five, four, six, seven, one, eight, zero,
Nearest to for: after, lal, peninsular, without, ldp, nkvd, rattus, monterrey,
Nearest to at: in, near, during, headlined, ducati, around, goblin, mgh,
Nearest to history: list, scorpions, tradition, heritage, period, dictionary, medieval, graduates,
Average loss at step 192000: 2.628245
Average loss at step 194000: 3.026561
Average loss at step 196000: 2.984893
Average loss at step 198000: 3.143168
Average loss at step 200000: 3.205368
Nearest to this: which, another, it, physicalism, the, that, some, adventists,
Nearest to more: less, most, greater, very, rather, better, increasingly, quite,
Nearest to system: systems, program, tarr, scheme, method, unit, popularize, platform,
Nearest to his: her, their, its, your, my, whose, our, the,
Nearest to while: although, though, however, when, after, but, began, before,
Nearest to after: before, during, despite, while, when, without, following, though,
Nearest to by: with, without, against, for, through, homemade, supranationalism, under,
Nearest to has: had, have, is, contains, since, was, holds, having,
Nearest to six: seven, five, eight, four, nine, three, zero, two,
Nearest to seven: six, eight, nine, five, four, three, zero, two,
Nearest to only: neither, approximately, either, astrophysical, strictly, usually, latter, mourns,
Nearest to were: are, was, have, had, these, those, while, include,
Nearest to two: three, four, five, six, seven, one, zero, eight,
Nearest to for: without, by, after, despite, against, parental, monterrey, hiker,
Nearest to at: near, in, during, goblin, headlined, within, spokane, mgh,
Nearest to history: list, tradition, scorpions, heritage, philosophy, science, period, encyclopedia,
Average loss at step 202000: 3.216425
Average loss at step 204000: 3.162979
Average loss at step 206000: 3.243221
Average loss at step 208000: 3.290432
Average loss at step 210000: 3.272263
Nearest to this: which, it, another, the, similar, physicalism, qibla, ruben,
Nearest to more: less, most, quite, greater, very, increasingly, smaller, better,
Nearest to system: systems, program, method, network, scheme, tarr, unit, process,
Nearest to his: her, their, its, my, your, our, whose, reckoning,
Nearest to while: although, when, though, after, however, before, if, despite,
Nearest to after: before, during, despite, while, following, when, without, since,
Nearest to by: through, from, under, against, supranationalism, sul, walther, during,
Nearest to has: had, have, is, was, contains, requires, uses, bord,
Nearest to six: seven, four, five, eight, three, nine, two, zero,
Nearest to seven: eight, six, four, five, three, nine, zero, two,
Nearest to only: approximately, eagerly, neither, strictly, either, keswick, universally, astrophysical,
Nearest to were: are, have, was, been, had, those, include, these,
Nearest to two: three, five, four, six, seven, eight, one, zero,
Nearest to for: without, parental, after, jingles, despite, nkvd, cranston, against,
Nearest to at: near, during, in, goblin, on, within, around, ih,
Nearest to history: list, scorpions, heritage, tradition, period, science, dictionary, wisest,
Average loss at step 212000: 3.262002
Average loss at step 214000: 3.151760
Average loss at step 216000: 3.085338
Average loss at step 218000: 3.245859
Average loss at step 220000: 3.223495
Nearest to this: which, it, the, that, another, fledging, some, kinetoscope,
Nearest to more: less, most, better, very, faster, quite, smaller, rather,
Nearest to system: systems, network, platform, program, scheme, methods, services, district,
Nearest to his: her, their, my, your, its, our, whose, reckoning,
Nearest to while: although, when, though, however, after, where, before, if,
Nearest to after: before, during, despite, when, without, within, while, until,
Nearest to by: through, without, using, supranationalism, with, when, from, exacted,
Nearest to has: had, have, contains, is, having, was, requires, bord,
Nearest to six: seven, five, eight, four, nine, three, zero, two,
Nearest to seven: eight, six, four, five, nine, three, zero, two,
Nearest to only: approximately, neither, vectors, trimester, eagerly, until, strictly, jury,
Nearest to were: are, those, was, have, these, be, being, although,
Nearest to two: three, four, five, six, seven, one, zero, eight,
Nearest to for: without, and, if, underscore, parental, unassigned, monterrey, collation,
Nearest to at: near, during, in, around, on, virtually, within, chardin,
Nearest to history: scorpions, tradition, list, heritage, science, galatian, graduates, literature,
Average loss at step 222000: 3.247897
Average loss at step 224000: 3.235916
Average loss at step 226000: 3.283499
Average loss at step 228000: 3.211165
Average loss at step 230000: 3.196982
Nearest to this: it, which, vdp, some, another, the, speeding, audita,
Nearest to more: less, most, better, quite, greater, faster, very, smaller,
Nearest to system: systems, platform, network, program, scheme, services, revenue, excelling,
Nearest to his: her, their, your, my, its, whose, strat, our,
Nearest to while: although, though, when, after, however, before, including, daytime,
Nearest to after: before, during, despite, when, while, without, following, within,
Nearest to by: through, for, including, with, exacted, in, via, without,
Nearest to has: had, have, is, was, contains, having, bord, since,
Nearest to six: four, eight, seven, five, three, nine, two, zero,
Nearest to seven: eight, six, four, five, three, nine, one, zero,
Nearest to only: approximately, neither, astrophysical, solved, jury, all, anglicanism, characterize,
Nearest to were: are, have, was, those, tend, these, although, had,
Nearest to two: three, four, five, one, six, seven, zero, eight,
Nearest to for: without, after, including, despite, of, by, parental, against,
Nearest to at: during, near, within, around, goblin, despite, on, baleen,
Nearest to history: list, scorpions, heritage, period, philosophy, graduates, cpsu, biography,
Average loss at step 232000: 2.925301
Average loss at step 234000: 3.094765
Average loss at step 236000: 3.194770
Average loss at step 238000: 3.059705
Average loss at step 240000: 3.178271
Nearest to this: it, which, vdp, due, another, kinetoscope, the, bellows,
Nearest to more: less, most, better, greater, quite, rather, smaller, very,
Nearest to system: systems, platform, network, scheme, program, revenue, colloidal, services,
Nearest to his: her, their, your, my, its, the, our, whose,
Nearest to while: although, though, when, after, however, before, whilst, began,
Nearest to after: before, despite, during, while, when, without, for, until,
Nearest to by: through, with, during, from, supranationalism, newly, multinational, numan,
Nearest to has: had, have, is, contains, having, maintains, was, requires,
Nearest to six: eight, seven, four, five, nine, three, two, zero,
Nearest to seven: eight, five, six, four, nine, three, zero, two,
Nearest to only: approximately, neither, best, astrophysical, adversely, eagerly, illegally, jury,
Nearest to were: are, was, have, had, those, be, these, been,
Nearest to two: three, four, five, six, seven, eight, one, zero,
Nearest to for: against, after, despite, without, before, to, at, during,
Nearest to at: in, during, near, within, on, goblin, for, under,
Nearest to history: list, period, career, heritage, philosophy, tradition, scorpions, editors,
Average loss at step 242000: 3.284501
Average loss at step 244000: 3.229692
Average loss at step 246000: 3.145926
Average loss at step 248000: 3.124782
Average loss at step 250000: 3.235253
Nearest to this: it, which, some, the, kinetoscope, that, another, qibla,
Nearest to more: less, most, smaller, very, quite, faster, rather, better,
Nearest to system: systems, platform, network, revenue, process, flatted, program, yar,
Nearest to his: her, their, its, your, my, our, the, reckoning,
Nearest to while: although, though, when, after, however, before, if, began,
Nearest to after: before, despite, during, while, when, without, afterwards, began,
Nearest to by: through, during, without, against, with, walther, via, exacted,
Nearest to has: had, have, contains, is, requires, was, having, includes,
Nearest to six: four, seven, five, eight, nine, three, two, zero,
Nearest to seven: four, eight, five, six, nine, three, zero, two,
Nearest to only: approximately, either, adversely, eagerly, jury, actually, best, solved,
Nearest to were: are, was, those, have, had, tend, remain, these,
Nearest to two: three, five, four, six, eight, seven, one, zero,
Nearest to for: without, parental, nkvd, including, despite, mclachlan, against, jingles,
Nearest to at: near, during, in, within, viewable, spacesuit, nair, fets,
Nearest to history: list, scorpions, career, period, graduates, timeline, heritage, tradition,
Average loss at step 252000: 3.272143
Average loss at step 254000: 3.279479
Average loss at step 256000: 3.184488
Average loss at step 258000: 3.135047
Average loss at step 260000: 3.204882
Nearest to this: it, which, the, another, what, any, vdp, adventists,
Nearest to more: less, most, rather, very, increasingly, greater, quite, larger,
Nearest to system: systems, platform, network, flatted, process, program, excelling, mobility,
Nearest to his: her, their, your, my, its, our, whose, personal,
Nearest to while: although, though, when, however, after, before, unlike, including,
Nearest to after: before, despite, during, when, while, without, following, since,
Nearest to by: during, through, including, with, without, and, walther, under,
Nearest to has: had, have, contains, is, was, maintains, having, requires,
Nearest to six: five, seven, three, eight, four, nine, zero, two,
Nearest to seven: eight, six, nine, five, four, three, zero, two,
Nearest to only: approximately, jury, adversely, eagerly, always, astrophysical, anglicanism, either,
Nearest to were: are, those, have, had, was, remain, these, been,
Nearest to two: three, four, six, five, seven, one, zero, isbn,
Nearest to for: against, despite, after, without, including, during, parental, from,
Nearest to at: near, during, in, within, despite, grammy, nobel, nair,
Nearest to history: list, scorpions, heritage, graduates, resources, dkk, tradition, bony,
Average loss at step 262000: 3.125470
Average loss at step 264000: 3.155089
Average loss at step 266000: 3.143880
Average loss at step 268000: 3.105616
Average loss at step 270000: 3.070938
Nearest to this: it, which, bloomsbury, another, each, any, what, vdp,
Nearest to more: less, most, increasingly, quite, very, rather, greater, better,
Nearest to system: systems, platform, network, method, excelling, process, program, district,
Nearest to his: her, their, my, your, our, its, whose, the,
Nearest to while: although, though, when, if, before, where, whilst, unless,
Nearest to after: before, despite, during, when, without, within, while, afterwards,
Nearest to by: through, during, against, without, from, in, with, inflorescence,
Nearest to has: had, contains, have, is, having, was, maintains, requires,
Nearest to six: eight, seven, five, four, three, nine, zero, two,
Nearest to seven: eight, six, five, four, nine, three, zero, two,
Nearest to only: always, astrophysical, mourns, strictly, jury, adversely, until, equally,
Nearest to were: are, was, had, have, remain, those, although, being,
Nearest to two: three, four, six, five, zero, seven, eight, nine,
Nearest to for: despite, against, without, after, underscore, during, intelligentsia, nkvd,
Nearest to at: near, in, during, within, goblin, fets, attended, unlucky,
Nearest to history: list, tradition, scorpions, origins, heritage, politics, period, philosophy,
Average loss at step 272000: 3.090658
Average loss at step 274000: 3.168533
Average loss at step 276000: 3.160430
Average loss at step 278000: 3.131688
Average loss at step 280000: 3.149287
Nearest to this: which, it, another, the, any, instead, what, similar,
Nearest to more: less, most, very, increasingly, rather, quite, greater, better,
Nearest to system: systems, platform, program, scheme, network, colloidal, excelling, convergence,
Nearest to his: her, their, my, your, its, our, whose, strat,
Nearest to while: although, though, when, however, began, but, before, despite,
Nearest to after: before, despite, during, when, without, afterwards, while, within,
Nearest to by: through, supranationalism, without, against, using, multinational, walther, reduces,
Nearest to has: had, have, is, contains, having, was, requires, maintains,
Nearest to six: eight, four, seven, five, nine, three, zero, two,
Nearest to seven: eight, six, nine, five, four, three, zero, two,
Nearest to only: jury, eagerly, approximately, strictly, adversely, universally, actually, platonist,
Nearest to were: are, was, had, those, have, became, these, esau,
Nearest to two: three, four, six, seven, eight, five, nine, one,
Nearest to for: despite, parental, without, while, although, including, after, during,
Nearest to at: near, during, in, under, within, goblin, headlined, fets,
Nearest to history: list, scorpions, graduates, heritage, champions, origins, cpsu, dkk,
Average loss at step 282000: 3.165362
Average loss at step 284000: 3.193754
Average loss at step 286000: 3.179881
Average loss at step 288000: 3.176425
Average loss at step 290000: 3.125511
Nearest to this: which, it, another, due, what, thanksgiving, that, chronos,
Nearest to more: less, most, greater, very, smaller, quite, larger, rather,
Nearest to system: systems, platform, program, network, equipment, scheme, excelling, facilities,
Nearest to his: her, their, my, your, its, our, whose, congratulations,
Nearest to while: although, though, when, began, however, whilst, where, despite,
Nearest to after: before, despite, during, without, when, within, afterwards, upon,
Nearest to by: through, with, against, multinational, newly, using, was, while,
Nearest to has: had, have, is, contains, maintains, was, having, provides,
Nearest to six: four, eight, five, seven, three, nine, two, zero,
Nearest to seven: eight, six, nine, five, four, three, zero, two,
Nearest to only: eagerly, jury, reversed, exec, approximately, platonist, strictly, anglicanism,
Nearest to were: are, was, have, had, tend, been, those, became,
Nearest to two: three, four, six, five, one, seven, eight, zero,
Nearest to for: despite, after, neq, parental, of, detonator, nkvd, luening,
Nearest to at: near, on, within, goblin, in, during, baleen, czechoslovak,
Nearest to history: list, scorpions, geography, politics, heritage, mali, tradition, overview,
Average loss at step 292000: 3.006332
Average loss at step 294000: 3.167324
Average loss at step 296000: 3.183997
Average loss at step 298000: 3.182282
Average loss at step 300000: 3.185168
Nearest to this: which, it, another, the, each, adventists, arising, instead,
Nearest to more: less, most, quite, very, greater, better, fewer, fairly,
Nearest to system: systems, scheme, platform, excelling, network, method, gesture, program,
Nearest to his: her, their, my, your, its, our, the, strat,
Nearest to while: although, when, though, whilst, began, however, where, unless,
Nearest to after: before, despite, without, during, when, afterwards, sportsmanship, upon,
Nearest to by: through, exacted, without, was, for, against, kool, wahhabis,
Nearest to has: had, have, is, contains, was, having, since, holds,
Nearest to six: eight, four, five, seven, nine, three, two, zero,
Nearest to seven: eight, six, four, nine, five, three, zero, two,
Nearest to only: eagerly, strictly, either, actually, neither, jury, nowhere, anglicanism,
Nearest to were: are, was, tend, those, have, remain, although, had,
Nearest to two: three, four, five, six, one, seven, eight, nine,
Nearest to for: despite, against, in, including, hiker, by, during, through,
Nearest to at: near, headlined, in, on, within, fets, unlucky, goblin,
Nearest to history: list, scorpions, tradition, europe, heritage, reign, prose, period,
Average loss at step 302000: 3.121063
Average loss at step 304000: 3.148821
Average loss at step 306000: 3.187077
Average loss at step 308000: 3.176108
Average loss at step 310000: 3.187011
Nearest to this: which, it, another, some, stratum, there, vdp, fledging,
Nearest to more: less, most, rather, quite, increasingly, greater, fewer, highly,
Nearest to system: systems, platform, network, scheme, program, facilities, method, excelling,
Nearest to his: their, her, my, your, its, whose, our, strat,
Nearest to while: although, when, after, though, whilst, before, unless, began,
Nearest to after: before, despite, during, while, when, without, afterwards, upon,
Nearest to by: through, against, using, including, exacted, supranationalism, under, multinational,
Nearest to has: had, have, is, contains, having, was, bord, maintains,
Nearest to six: eight, four, five, nine, three, seven, two, zero,
Nearest to seven: eight, six, five, nine, four, three, zero, two,
Nearest to only: actually, always, strictly, eagerly, jury, universally, adversely, neither,
Nearest to were: are, was, those, have, had, remain, although, tend,
Nearest to two: three, four, five, one, six, seven, zero, eight,
Nearest to for: despite, against, including, mclachlan, parental, in, nkvd, without,
Nearest to at: near, headlined, in, on, fets, during, lombardo, goblin,
Nearest to history: list, scorpions, overview, phonology, origins, mali, period, lists,
Average loss at step 312000: 3.148970
Average loss at step 314000: 3.146442
Average loss at step 316000: 2.948270
Average loss at step 318000: 3.047773
Average loss at step 320000: 3.171752
Nearest to this: which, it, there, another, vdp, audita, what, chalmers,
Nearest to more: less, most, increasingly, highly, greater, quite, very, better,
Nearest to system: systems, network, scheme, platform, district, program, method, unit,
Nearest to his: her, their, my, your, its, our, yearning, strat,
Nearest to while: although, though, when, before, after, whilst, unlike, however,
Nearest to after: before, despite, during, when, while, without, unless, afterwards,
Nearest to by: through, using, including, supranationalism, exacted, fillmore, besides, from,
Nearest to has: had, have, is, contains, maintains, was, having, bord,
Nearest to six: five, eight, four, seven, three, nine, two, zero,
Nearest to seven: six, five, eight, nine, four, three, zero, two,
Nearest to only: actually, platonist, always, neither, strictly, until, jury, universally,
Nearest to were: are, was, have, those, tend, been, remain, these,
Nearest to two: three, six, four, five, eight, seven, one, nine,
Nearest to for: including, without, of, parental, underlying, despite, against, mclachlan,
Nearest to at: lombardo, near, headlined, in, baleen, during, grammy, within,
Nearest to history: list, politics, overview, phonology, demographics, reductionist, scorpions, mali,
Average loss at step 322000: 3.154612
Average loss at step 324000: 3.123861
Average loss at step 326000: 3.156028
Average loss at step 328000: 3.146520
Average loss at step 330000: 3.081344
Nearest to this: which, it, another, audita, instead, vdp, similar, spontaneous,
Nearest to more: less, very, most, fewer, quite, increasingly, fairly, highly,
Nearest to system: systems, scheme, network, program, platform, technique, methods, method,
Nearest to his: her, their, my, its, your, our, congratulations, dividends,
Nearest to while: although, though, when, after, including, whilst, and, before,
Nearest to after: before, despite, during, when, while, without, thereafter, afterwards,
Nearest to by: through, from, using, multinational, via, during, against, including,
Nearest to has: had, have, contains, having, is, requires, bord, holds,
Nearest to six: seven, five, four, eight, nine, three, two, zero,
Nearest to seven: six, five, eight, nine, four, three, two, one,
Nearest to only: actually, always, jury, eagerly, cameo, best, adversely, neither,
Nearest to were: are, was, have, had, those, tend, remain, these,
Nearest to two: three, six, four, five, seven, one, eight, nine,
Nearest to for: against, without, despite, from, and, after, valerian, in,
Nearest to at: in, near, during, headlined, goblin, unlucky, furigana, grammy,
Nearest to history: list, politics, origins, overview, biography, lists, philosophy, prose,
Average loss at step 332000: 3.151782
Average loss at step 334000: 3.154220
Average loss at step 336000: 3.200727
Average loss at step 338000: 3.204378
Average loss at step 340000: 3.185070
Nearest to this: which, it, another, the, any, what, some, each,
Nearest to more: less, most, fewer, very, better, greater, increasingly, fairly,
Nearest to system: systems, platform, scheme, network, program, methods, node, cpu,
Nearest to his: her, their, my, your, its, our, the, whose,
Nearest to while: although, though, when, whilst, unlike, unless, however, before,
Nearest to after: before, during, when, despite, without, while, until, thereafter,
Nearest to by: through, without, when, wahhabis, using, including, via, from,
Nearest to has: had, have, is, having, requires, contains, was, makes,
Nearest to six: eight, seven, five, nine, four, three, zero, two,
Nearest to seven: five, eight, nine, four, six, three, zero, two,
Nearest to only: actually, neither, jury, always, compromised, universally, nowhere, strictly,
Nearest to were: are, was, have, tend, those, these, had, although,
Nearest to two: three, four, six, one, seven, five, eight, nine,
Nearest to for: including, against, despite, without, detonator, savants, during, underscore,
Nearest to at: in, headlined, baleen, near, unlucky, during, waals, spacesuit,
Nearest to history: list, politics, origins, prose, continuation, rico, bony, timeline,
Average loss at step 342000: 3.161397
Average loss at step 344000: 3.182237
Average loss at step 346000: 3.192699
Average loss at step 348000: 3.064527
Average loss at step 350000: 3.152178
Nearest to this: it, which, another, some, the, that, any, metadata,
Nearest to more: less, most, very, fewer, faster, better, larger, smaller,
Nearest to system: systems, platform, scheme, network, program, flatted, model, cpu,
Nearest to his: her, their, my, your, our, its, strat, yearning,
Nearest to while: although, though, when, unless, whilst, before, after, however,
Nearest to after: before, despite, during, when, without, while, until, afterwards,
Nearest to by: through, supranationalism, including, when, from, wahhabis, sonnet, gynt,
Nearest to has: had, have, is, contains, requires, having, was, holds,
Nearest to six: seven, five, eight, four, nine, three, zero, two,
Nearest to seven: nine, eight, six, five, four, three, zero, one,
Nearest to only: jury, actually, eagerly, neither, reciprocated, adversely, strictly, always,
Nearest to were: are, was, have, these, had, tend, those, remain,
Nearest to two: four, three, one, five, six, zero, eight, seven,
Nearest to for: with, including, of, in, despite, parental, mclachlan, although,
Nearest to at: headlined, baleen, tristar, in, unlucky, during, fets, near,
Nearest to history: list, origins, politics, overview, timeline, continuation, lists, geography,
Average loss at step 352000: 3.053453
Average loss at step 354000: 3.188912
Average loss at step 356000: 3.137085
Average loss at step 358000: 3.124790
Average loss at step 360000: 3.200336
Nearest to this: which, it, vdp, adventists, qibla, actually, what, another,
Nearest to more: less, most, very, increasingly, fewer, quite, fairly, extremely,
Nearest to system: systems, platform, program, network, scheme, isuzu, district, model,
Nearest to his: her, their, its, my, your, our, the, s,
Nearest to while: although, though, when, whilst, however, but, unless, began,
Nearest to after: before, despite, during, when, without, through, while, afterwards,
Nearest to by: through, with, from, including, against, without, besides, fillmore,
Nearest to has: had, have, is, maintains, holds, requires, was, having,
Nearest to six: seven, four, five, eight, three, nine, zero, two,
Nearest to seven: six, eight, five, four, nine, three, zero, one,
Nearest to only: actually, jury, reciprocated, adversely, theoretically, eagerly, strictly, replacement,
Nearest to were: are, had, have, was, those, these, remain, arrived,
Nearest to two: three, four, five, six, one, seven, zero, eight,
Nearest to for: against, without, including, after, unreachable, hiker, despite, cotangent,
Nearest to at: during, in, within, headlined, near, grammy, on, czechoslovak,
Nearest to history: list, origins, comparison, continuation, overview, politics, timeline, scorpions,
Average loss at step 362000: 3.160987
Average loss at step 364000: 3.005450
Average loss at step 366000: 3.138097
Average loss at step 368000: 3.136010
Average loss at step 370000: 3.141057
Nearest to this: which, the, it, kinetoscope, vdp, any, each, another,
Nearest to more: less, quite, most, very, increasingly, fairly, fewer, smaller,
Nearest to system: systems, platform, network, program, method, scheme, unit, practice,
Nearest to his: her, their, my, your, its, our, whose, personal,
Nearest to while: although, though, when, after, whilst, however, whereas, unless,
Nearest to after: before, during, despite, when, while, without, afterwards, within,
Nearest to by: through, without, from, using, when, kool, via, scarcely,
Nearest to has: had, have, is, maintains, requires, contains, bord, having,
Nearest to six: seven, four, five, eight, nine, three, two, zero,
Nearest to seven: five, six, nine, eight, four, three, zero, two,
Nearest to only: actually, jury, approximately, eagerly, universally, adversely, strictly, always,
Nearest to were: are, was, had, have, those, remain, became, been,
Nearest to two: three, four, six, five, one, seven, eight, nine,
Nearest to for: without, in, mclachlan, against, including, after, aafc, of,
Nearest to at: in, during, fets, near, baleen, goblin, within, waals,
Nearest to history: list, origins, politics, biography, timeline, continuation, galatian, overview,
Average loss at step 372000: 3.164176
Average loss at step 374000: 3.145629
Average loss at step 376000: 3.155063
Average loss at step 378000: 3.088891
Average loss at step 380000: 3.022311
Nearest to this: which, it, similar, another, these, vdp, some, speeding,
Nearest to more: less, most, quite, very, fewer, increasingly, fairly, greater,
Nearest to system: systems, network, platform, program, district, scheme, terminology, facilities,
Nearest to his: her, their, its, my, your, the, our, whose,
Nearest to while: although, though, whilst, when, however, including, began, after,
Nearest to after: before, despite, during, when, following, without, while, afterwards,
Nearest to by: through, including, opposes, while, exacted, with, via, ost,
Nearest to has: had, have, is, maintains, having, contains, was, bord,
Nearest to six: four, five, seven, eight, three, nine, two, zero,
Nearest to seven: six, eight, five, four, nine, three, zero, two,
Nearest to only: jury, actually, strictly, reciprocated, eagerly, neither, theoretically, supposed,
Nearest to were: are, was, have, remain, had, include, be, those,
Nearest to two: three, four, six, five, seven, one, eight, nine,
Nearest to for: against, without, including, unreachable, parental, from, despite, after,
Nearest to at: near, lombardo, during, in, under, within, baleen, headlined,
Nearest to history: list, biography, origins, timeline, politics, lists, heritage, continuation,
Average loss at step 382000: 3.155192
Average loss at step 384000: 3.029047
Average loss at step 386000: 3.077719
Average loss at step 388000: 3.137079
Average loss at step 390000: 3.007795
Nearest to this: which, it, any, another, some, the, each, qibla,
Nearest to more: less, fewer, quite, most, very, greater, fairly, increasingly,
Nearest to system: systems, network, platform, unit, program, scheme, district, model,
Nearest to his: her, their, my, its, your, whose, our, the,
Nearest to while: although, though, when, whilst, however, unlike, unless, after,
Nearest to after: before, despite, during, when, without, while, following, thereafter,
Nearest to by: without, through, via, from, including, besides, when, supranationalism,
Nearest to has: had, have, is, contains, was, having, requires, maintains,
Nearest to six: seven, five, four, eight, three, nine, two, zero,
Nearest to seven: six, five, four, eight, nine, three, zero, two,
Nearest to only: jury, actually, without, strictly, cameo, until, grotto, reciprocated,
Nearest to were: are, was, include, those, have, tend, seem, lindo,
Nearest to two: three, four, five, six, seven, one, zero, eight,
Nearest to for: despite, against, without, underscore, including, after, hiker, of,
Nearest to at: near, in, goblin, lombardo, under, headlined, during, baleen,
Nearest to history: list, origins, heritage, politics, biography, continuation, timeline, overview,
Average loss at step 392000: 3.199319
Average loss at step 394000: 3.142022
Average loss at step 396000: 3.132411
Average loss at step 398000: 2.890836
Average loss at step 400000: 3.185430
Nearest to this: which, it, another, similar, deterrence, qibla, adventists, vdp,
Nearest to more: less, most, fewer, very, better, longer, quite, worse,
Nearest to system: systems, network, platform, program, scheme, district, unit, password,
Nearest to his: her, their, my, your, its, our, whose, childhood,
Nearest to while: although, though, when, whilst, however, whereas, unlike, including,
Nearest to after: before, during, despite, when, following, while, without, afterwards,
Nearest to by: through, via, locate, without, using, opposes, kool, numan,
Nearest to has: had, have, is, was, contains, maintains, includes, requires,
Nearest to six: seven, five, eight, four, three, nine, two, zero,
Nearest to seven: eight, six, five, nine, four, three, two, zero,
Nearest to only: actually, adversely, theoretically, already, approximately, nowhere, eagerly, compromised,
Nearest to were: are, have, was, had, those, seem, these, tend,
Nearest to two: three, five, four, six, one, seven, eight, zero,
Nearest to for: of, darnell, toolkit, inf, without, mclachlan, nkvd, after,
Nearest to at: near, in, lombardo, during, fets, on, spacesuit, baleen,
Nearest to history: list, origins, timeline, politics, heritage, geography, continuation, presocratic,
Average loss at step 402000: 3.148692
Average loss at step 404000: 3.158265
Average loss at step 406000: 3.171251
Average loss at step 408000: 3.179768
Average loss at step 410000: 3.133118
Nearest to this: which, it, another, instead, each, that, spontaneous, the,
Nearest to more: less, very, quite, most, fairly, better, increasingly, extremely,
Nearest to system: systems, platform, unit, program, scheme, process, method, practice,
Nearest to his: her, their, my, your, its, our, whose, congratulations,
Nearest to while: although, though, when, whilst, after, before, however, unlike,
Nearest to after: before, during, despite, when, while, thereafter, following, without,
Nearest to by: during, via, through, when, against, guillotined, wahhabis, including,
Nearest to has: had, have, is, contains, having, was, maintains, requires,
Nearest to six: eight, seven, five, four, nine, three, two, zero,
Nearest to seven: six, eight, five, nine, four, three, zero, two,
Nearest to only: eagerly, actually, adversely, nowhere, compromised, theoretically, reciprocated, strictly,
Nearest to were: are, was, have, had, those, include, tend, remain,
Nearest to two: three, four, six, five, eight, seven, one, zero,
Nearest to for: when, after, without, while, during, including, despite, mclachlan,
Nearest to at: near, in, during, headlined, goblin, under, lombardo, waals,
Nearest to history: list, geography, heritage, scorpions, mali, armenia, timeline, biography,
Average loss at step 412000: 3.011284
Average loss at step 414000: 3.129965
Average loss at step 416000: 3.179684
Average loss at step 418000: 3.129467
Average loss at step 420000: 3.094569
Nearest to this: which, it, another, each, deterrence, qibla, latter, vdp,
Nearest to more: less, very, quite, most, fairly, smaller, fewer, better,
Nearest to system: systems, platform, scheme, unit, practice, district, services, suppression,
Nearest to his: her, their, my, your, its, whose, our, playfulness,
Nearest to while: although, though, whilst, when, however, but, whereas, unlike,
Nearest to after: before, during, despite, when, following, while, thereafter, afterwards,
Nearest to by: through, from, including, fillmore, in, during, besides, levin,
Nearest to has: had, have, having, is, contains, was, maintains, requires,
Nearest to six: eight, seven, five, nine, four, three, zero, two,
Nearest to seven: eight, six, nine, five, four, three, zero, two,
Nearest to only: eagerly, adversely, strictly, nowhere, mourns, actually, either, jury,
Nearest to were: are, was, have, those, had, include, remain, seem,
Nearest to two: three, four, five, six, seven, eight, one, nine,
Nearest to for: without, djamena, after, mclachlan, when, during, cotangent, despite,
Nearest to at: near, in, goblin, fets, headlined, during, university, beside,
Nearest to history: list, philosophy, timeline, evolution, heritage, politics, geography, tradition,
Average loss at step 422000: 3.145522
Average loss at step 424000: 3.145220
Average loss at step 426000: 3.142232
Average loss at step 428000: 3.155569
Average loss at step 430000: 3.092519
Nearest to this: which, it, another, physicalism, there, adventists, some, each,
Nearest to more: less, most, quite, fewer, very, smaller, greater, longer,
Nearest to system: systems, scheme, platform, network, district, process, infrastructure, boulevards,
Nearest to his: her, their, your, its, my, our, whose, yearning,
Nearest to while: although, though, whilst, however, when, whereas, unlike, despite,
Nearest to after: before, during, despite, when, afterwards, following, within, thereafter,
Nearest to by: through, via, including, fillmore, locate, involves, opposes, from,
Nearest to has: had, have, is, was, having, maintains, contains, enjoys,
Nearest to six: eight, seven, four, five, nine, three, two, zero,
Nearest to seven: eight, six, nine, five, four, three, zero, two,
Nearest to only: eagerly, adversely, strictly, grotto, mourns, approximately, jury, mid,
Nearest to were: are, have, was, those, these, had, seem, including,
Nearest to two: three, four, six, seven, eight, five, one, zero,
Nearest to for: despite, including, of, mclachlan, parental, intelligentsia, djamena, to,
Nearest to at: near, in, within, during, goblin, on, licking, fets,
Nearest to history: list, evolution, timeline, scorpions, origins, overview, geography, darmstadt,
Average loss at step 432000: 3.025622
Average loss at step 434000: 3.146400
Average loss at step 436000: 3.088474
Average loss at step 438000: 3.138083
Average loss at step 440000: 3.120109
Nearest to this: which, it, another, some, the, due, larps, there,
Nearest to more: less, most, quite, fewer, very, smaller, better, fairly,
Nearest to system: systems, scheme, network, process, platform, method, boulevards, suppression,
Nearest to his: her, their, its, your, my, our, yearning, reckoning,
Nearest to while: although, however, whilst, though, whereas, when, unlike, but,
Nearest to after: before, during, despite, when, without, while, afterwards, thereafter,
Nearest to by: when, through, from, via, without, wozniak, scarcely, fillmore,
Nearest to has: had, have, is, requires, was, contains, provides, holds,
Nearest to six: eight, four, five, nine, seven, three, zero, two,
Nearest to seven: five, eight, nine, four, six, three, zero, two,
Nearest to only: eagerly, strictly, jury, eurozone, adversely, primarily, even, platonist,
Nearest to were: are, was, these, had, those, have, remain, tend,
Nearest to two: three, four, five, one, six, seven, eight, zero,
Nearest to for: of, without, in, collation, by, to, djamena, mclachlan,
Nearest to at: near, during, within, in, fets, goblin, waals, on,
Nearest to history: list, timeline, evolution, origins, heritage, overview, scorpions, use,
Average loss at step 442000: 3.119235
Average loss at step 444000: 2.960221
Average loss at step 446000: 2.557209
Average loss at step 448000: 2.884939
Average loss at step 450000: 2.840470
Nearest to this: which, another, it, adventists, physicalism, thanksgiving, some, qibla,
Nearest to more: less, most, quite, fewer, very, smaller, better, greater,
Nearest to system: systems, scheme, method, platform, unit, process, district, network,
Nearest to his: her, their, its, your, my, the, our, whose,
Nearest to while: although, whilst, though, when, after, however, before, but,
Nearest to after: before, despite, during, when, without, while, afterwards, following,
Nearest to by: from, through, including, during, when, via, under, for,
Nearest to has: had, have, is, holds, requires, having, maintains, bord,
Nearest to six: seven, eight, four, five, nine, three, zero, two,
Nearest to seven: six, eight, five, four, nine, three, zero, two,
Nearest to only: eagerly, mourns, adversely, eurozone, cameo, jury, strictly, last,
Nearest to were: are, include, have, these, those, seem, was, remain,
Nearest to two: three, five, four, six, eight, seven, zero, one,
Nearest to for: after, without, by, aafc, during, djamena, from, monterrey,
Nearest to at: in, near, during, on, beside, within, headlined, from,
Nearest to history: list, scorpions, tradition, heritage, timeline, origins, course, geography,
Average loss at step 452000: 3.043951
Average loss at step 454000: 3.089754
Average loss at step 456000: 3.101047
Average loss at step 458000: 3.028919
Average loss at step 460000: 3.132504
Nearest to this: it, which, another, physicalism, adventists, qibla, the, larps,
Nearest to more: less, most, quite, increasingly, smaller, very, fewer, fairly,
Nearest to system: systems, scheme, method, unit, network, platform, practice, district,
Nearest to his: her, their, my, its, your, whose, our, the,
Nearest to while: although, whilst, when, though, after, however, before, despite,
Nearest to after: before, despite, while, during, following, when, afterwards, without,
Nearest to by: including, with, from, through, via, besides, during, without,
Nearest to has: had, have, is, holds, requires, was, maintains, contains,
Nearest to six: five, four, eight, seven, nine, three, zero, two,
Nearest to seven: eight, four, six, five, nine, three, zero, two,
Nearest to only: eagerly, mourns, adversely, either, strictly, grotto, without, jury,
Nearest to were: are, was, include, remain, had, those, tend, have,
Nearest to two: three, four, five, six, one, seven, eight, zero,
Nearest to for: against, without, kamov, by, genocidal, lal, djamena, paulette,
Nearest to at: near, during, in, within, goblin, nair, outside, attended,
Nearest to history: list, tradition, heritage, scorpions, origins, course, timeline, archaeology,
Average loss at step 462000: 3.151565
Average loss at step 464000: 3.139610
Average loss at step 466000: 3.153025
Average loss at step 468000: 2.998199
Average loss at step 470000: 3.015470
Nearest to this: which, it, the, another, qibla, physicalism, his, some,
Nearest to more: less, most, fewer, smaller, quite, greater, better, increasingly,
Nearest to system: systems, platform, scheme, network, practice, district, method, popularize,
Nearest to his: her, their, my, its, your, the, our, whose,
Nearest to while: although, whilst, though, when, however, after, whereas, including,
Nearest to after: before, during, despite, when, without, afterwards, while, following,
Nearest to by: through, wozniak, supranationalism, via, with, walpole, kool, numan,
Nearest to has: had, have, maintains, holds, is, having, was, requires,
Nearest to six: five, eight, seven, four, three, nine, two, zero,
Nearest to seven: eight, four, six, five, nine, three, zero, two,
Nearest to only: eagerly, mourns, eurozone, until, neither, last, adversely, best,
Nearest to were: are, was, have, those, remain, had, tend, these,
Nearest to two: three, four, five, six, one, seven, eight, zero,
Nearest to for: without, against, genocidal, djamena, jingles, underscore, pushkin, slava,
Nearest to at: near, during, goblin, attended, in, on, nair, observes,
Nearest to history: list, scorpions, heritage, tradition, origins, timeline, course, geography,
Average loss at step 472000: 3.128400
Average loss at step 474000: 3.090059
Average loss at step 476000: 3.118545
Average loss at step 478000: 3.109953
Average loss at step 480000: 3.171181
Nearest to this: which, it, the, there, some, another, qibla, that,
Nearest to more: less, most, quite, fewer, better, fairly, faster, greater,
Nearest to system: systems, platform, scheme, method, network, terminology, procedure, process,
Nearest to his: her, their, your, my, its, the, our, whose,
Nearest to while: although, whilst, though, when, however, whereas, moreover, unlike,
Nearest to after: before, during, despite, when, without, while, afterwards, thereafter,
Nearest to by: through, wozniak, without, walpole, for, via, supranationalism, levin,
Nearest to has: had, have, is, was, having, holds, requires, bord,
Nearest to six: four, eight, five, seven, nine, three, two, zero,
Nearest to seven: eight, four, six, nine, five, three, zero, two,
Nearest to only: adversely, actually, eagerly, strictly, necessarily, already, never, neither,
Nearest to were: are, was, have, include, those, these, including, seem,
Nearest to two: three, four, five, six, one, eight, seven, zero,
Nearest to for: after, without, consider, in, against, of, by, underscore,
Nearest to at: near, during, in, within, goblin, around, under, on,
Nearest to history: list, course, heritage, timeline, origins, overview, introduction, career,
Average loss at step 482000: 3.094646
Average loss at step 484000: 3.044305
Average loss at step 486000: 2.802755
Average loss at step 488000: 3.040465
Average loss at step 490000: 3.081246
Nearest to this: which, it, vdp, adventists, speeding, nhk, the, another,
Nearest to more: less, quite, most, greater, fewer, better, smaller, fairly,
Nearest to system: systems, scheme, platform, network, practice, terminology, procedure, microprocessor,
Nearest to his: her, their, my, your, its, the, our, congratulations,
Nearest to while: although, whilst, though, whereas, moreover, when, after, however,
Nearest to after: before, during, despite, when, while, without, afterwards, following,
Nearest to by: through, from, wozniak, supranationalism, in, besides, numan, gladys,
Nearest to has: had, have, maintains, is, having, requires, contains, holds,
Nearest to six: eight, four, seven, nine, five, three, zero, two,
Nearest to seven: four, five, nine, eight, six, three, zero, two,
Nearest to only: adversely, actually, eagerly, strictly, either, specifically, anglicanism, neither,
Nearest to were: are, those, have, was, these, had, came, tend,
Nearest to two: three, four, eight, five, one, six, seven, nine,
Nearest to for: against, after, if, despite, without, underscore, when, mclachlan,
Nearest to at: within, during, in, near, around, goblin, azteca, fets,
Nearest to history: list, timeline, overview, heritage, course, origins, tradition, geography,
Average loss at step 492000: 2.914950
Average loss at step 494000: 3.107833
Average loss at step 496000: 3.170928
Average loss at step 498000: 3.118737
Average loss at step 500000: 2.982122
Nearest to this: it, which, thanksgiving, some, adventists, each, qibla, kinetoscope,
Nearest to more: less, fewer, most, quite, better, faster, smaller, greater,
Nearest to system: systems, platform, network, container, microprocessor, controllers, format, scheme,
Nearest to his: her, their, my, your, its, the, our, jointly,
Nearest to while: although, whilst, when, though, after, whereas, before, however,
Nearest to after: before, during, despite, while, when, afterwards, without, until,
Nearest to by: through, via, numan, fillmore, during, with, walther, in,
Nearest to has: had, have, is, having, was, contains, includes, requires,
Nearest to six: eight, four, seven, nine, five, three, zero, two,
Nearest to seven: eight, four, six, five, nine, three, zero, two,
Nearest to only: adversely, cmyk, actually, replacement, jury, eurozone, first, approximately,
Nearest to were: are, was, those, have, these, had, include, including,
Nearest to two: three, four, five, eight, six, zero, seven, one,
Nearest to for: including, without, while, with, mclachlan, against, unreachable, during,
Nearest to at: in, near, within, during, on, around, goblin, tristar,
Nearest to history: list, timeline, career, heritage, tradition, sel, premier, scorpions,
Average loss at step 502000: 3.069266
Average loss at step 504000: 3.126124
Average loss at step 506000: 3.162306
Average loss at step 508000: 3.166463
Average loss at step 510000: 3.099300
Nearest to this: it, which, thanksgiving, adventists, whatever, qibla, what, vdp,
Nearest to more: less, most, fewer, quite, better, greater, faster, worse,
Nearest to system: systems, platform, practice, model, network, mobility, procedure, process,
Nearest to his: her, their, your, my, its, our, personal, lagged,
Nearest to while: whilst, although, though, when, however, began, whereas, including,
Nearest to after: before, during, despite, when, without, afterwards, while, since,
Nearest to by: through, including, without, during, via, provoke, manichaean, from,
Nearest to has: had, have, is, requires, contains, maintains, was, holds,
Nearest to six: seven, eight, five, four, nine, three, zero, two,
Nearest to seven: eight, five, six, four, nine, three, zero, one,
Nearest to only: adversely, actually, either, astrophysical, cmyk, eagerly, jury, sole,
Nearest to were: are, those, was, had, include, remain, tend, have,
Nearest to two: three, four, five, six, eight, seven, one, isbn,
Nearest to for: against, including, despite, after, during, darnell, consider, aafc,
Nearest to at: during, near, in, within, despite, goblin, gernika, viewable,
Nearest to history: list, timeline, overview, tradition, heritage, scorpions, origins, geography,
Average loss at step 512000: 3.043921
Average loss at step 514000: 3.121496
Average loss at step 516000: 3.010661
Average loss at step 518000: 3.100614
Average loss at step 520000: 3.061450
Nearest to this: it, which, the, vdp, some, bloomsbury, adventists, kinetoscope,
Nearest to more: less, most, quite, fewer, greater, very, better, worse,
Nearest to system: systems, network, platform, method, excelling, framework, scheme, bundesrat,
Nearest to his: her, their, its, your, my, our, personal, reckoning,
Nearest to while: although, whilst, when, though, whereas, however, moreover, including,
Nearest to after: before, during, despite, when, while, without, afterwards, unless,
Nearest to by: through, from, via, in, with, without, including, supranationalism,
Nearest to has: had, have, contains, is, requires, having, maintains, was,
Nearest to six: seven, eight, four, nine, five, three, zero, two,
Nearest to seven: eight, six, five, four, nine, three, zero, two,
Nearest to only: adversely, actually, either, always, eagerly, nowhere, jury, canteen,
Nearest to were: are, remain, was, those, had, have, seem, appear,
Nearest to two: three, four, six, five, seven, eight, zero, one,
Nearest to for: without, darnell, aafc, collation, after, orthonormal, vigilante, mclachlan,
Nearest to at: near, in, during, goblin, fets, within, waals, dye,
Nearest to history: list, tradition, heritage, origins, overview, timeline, archaeology, scorpions,
Average loss at step 522000: 3.027469
Average loss at step 524000: 2.953004
Average loss at step 526000: 3.036654
Average loss at step 528000: 3.084829
Average loss at step 530000: 3.077833
Nearest to this: which, it, adventists, instead, some, what, there, wilmut,
Nearest to more: less, most, fewer, quite, greater, better, very, fairly,
Nearest to system: systems, platform, method, network, excelling, framework, container, scheme,
Nearest to his: her, their, my, your, its, whose, our, the,
Nearest to while: although, whilst, though, when, whereas, however, began, after,
Nearest to after: before, during, despite, when, afterwards, without, thereafter, while,
Nearest to by: through, via, with, platt, against, multinational, levin, in,
Nearest to has: had, have, is, requires, contains, maintains, was, includes,
Nearest to six: eight, four, seven, five, three, nine, two, zero,
Nearest to seven: eight, six, four, five, nine, three, two, zero,
Nearest to only: adversely, actually, always, mourns, eagerly, jury, either, nowhere,
Nearest to were: are, was, those, had, have, seem, contain, remain,
Nearest to two: three, four, six, eight, five, seven, one, zero,
Nearest to for: after, including, pra, in, without, meagre, when, maus,
Nearest to at: in, near, during, fets, within, goblin, attended, headlined,
Nearest to history: list, heritage, tradition, champions, origins, timeline, overview, scorpions,
Average loss at step 532000: 3.049186
Average loss at step 534000: 3.082543
Average loss at step 536000: 3.083668
Average loss at step 538000: 3.108207
Average loss at step 540000: 3.110565
Nearest to this: which, it, what, thanksgiving, that, another, the, some,
Nearest to more: less, fewer, most, greater, better, smaller, quite, worse,
Nearest to system: systems, platform, network, scheme, procedure, cisc, abruzzo, controllers,
Nearest to his: her, their, my, your, its, our, whose, jointly,
Nearest to while: although, though, whilst, when, however, involves, began, unlike,
Nearest to after: before, despite, during, when, afterwards, without, thereafter, within,
Nearest to by: through, via, using, against, with, wahhabis, without, multinational,
Nearest to has: had, have, is, contains, was, having, requires, maintains,
Nearest to six: four, eight, seven, five, three, nine, two, zero,
Nearest to seven: eight, five, nine, four, six, three, one, two,
Nearest to only: adversely, eagerly, nowhere, platonist, mid, jury, strictly, primarily,
Nearest to were: are, was, had, have, became, seem, those, connexion,
Nearest to two: three, four, six, five, seven, one, eight, nine,
Nearest to for: after, against, of, meagre, detonator, collation, without, despite,
Nearest to at: near, within, in, during, goblin, lombardo, on, exceptionally,
Nearest to history: list, timeline, heritage, origins, overview, scorpions, champions, continuation,
Average loss at step 542000: 3.100714
Average loss at step 544000: 3.004686
Average loss at step 546000: 2.955586
Average loss at step 548000: 3.107247
Average loss at step 550000: 3.102408
Nearest to this: which, it, another, instead, thanksgiving, what, bloomsbury, some,
Nearest to more: less, most, fewer, quite, very, greater, better, fairly,
Nearest to system: systems, platform, scheme, network, gesture, suppression, method, container,
Nearest to his: their, her, my, your, its, our, whose, strat,
Nearest to while: although, whilst, when, though, however, began, after, involves,
Nearest to after: before, despite, during, when, afterwards, without, thereafter, while,
Nearest to by: through, using, against, with, without, via, kool, dimaggio,
Nearest to has: had, have, contains, is, maintains, holds, requires, having,
Nearest to six: five, eight, four, seven, three, nine, two, zero,
Nearest to seven: eight, five, six, nine, four, three, zero, one,
Nearest to only: either, trimester, mid, eagerly, adversely, nowhere, anglicanism, cameo,
Nearest to were: are, was, had, tend, have, although, became, remain,
Nearest to two: three, four, five, six, one, eight, seven, nine,
Nearest to for: including, against, toolkit, lal, interdiction, to, djamena, jingles,
Nearest to at: near, during, in, headlined, milken, lahoud, on, lombardo,
Nearest to history: list, timeline, overview, lists, scorpions, biography, heritage, politics,
Average loss at step 552000: 3.118540
Average loss at step 554000: 3.097284
Average loss at step 556000: 3.074201
Average loss at step 558000: 3.068424
Average loss at step 560000: 3.114250
Nearest to this: which, it, the, another, there, some, ambergris, arising,
Nearest to more: less, fewer, most, quite, increasingly, fairly, extremely, highly,
Nearest to system: systems, scheme, platform, method, network, gesture, procedure, abruzzo,
Nearest to his: her, their, my, your, its, our, whose, playfulness,
Nearest to while: whilst, although, when, though, however, including, after, despite,
Nearest to after: before, despite, during, when, while, afterwards, thereafter, without,
Nearest to by: through, against, wahhabis, via, involves, kool, wozniak, opposes,
Nearest to has: had, have, is, was, contains, having, maintains, holds,
Nearest to six: eight, four, five, nine, seven, three, two, zero,
Nearest to seven: eight, six, five, nine, four, zero, three, two,
Nearest to only: eagerly, adversely, either, anglicanism, mid, specifically, always, strictly,
Nearest to were: are, have, was, remain, tend, had, those, appear,
Nearest to two: three, five, four, one, six, eight, zero, nine,
Nearest to for: including, kamov, djamena, against, while, in, during, chrominance,
Nearest to at: near, during, unlucky, in, lahoud, headlined, topalov, waals,
Nearest to history: list, timeline, origins, lists, overview, scorpions, champions, heritage,
Average loss at step 562000: 3.110759
Average loss at step 564000: 3.111712
Average loss at step 566000: 3.073410
Average loss at step 568000: 3.083589
Average loss at step 570000: 2.835434
Nearest to this: which, it, there, that, the, arising, some, vdp,
Nearest to more: less, fewer, smaller, larger, most, greater, better, increasingly,
Nearest to system: systems, platform, network, scheme, district, rule, suppression, bundesrat,
Nearest to his: her, their, my, your, our, its, yearning, whose,
Nearest to while: whilst, although, though, when, including, after, before, unless,
Nearest to after: before, despite, during, when, while, without, following, afterwards,
Nearest to by: including, through, for, subtracting, junta, in, from, during,
Nearest to has: had, have, is, maintains, contains, was, provides, having,
Nearest to six: four, five, eight, seven, three, nine, two, zero,
Nearest to seven: six, eight, four, five, nine, three, two, one,
Nearest to only: actually, eagerly, always, until, mid, strictly, nowhere, anglicanism,
Nearest to were: are, was, have, remain, those, had, tend, kimmei,
Nearest to two: three, four, six, one, five, seven, eight, nine,
Nearest to for: including, against, by, of, despite, djamena, after, cotangent,
Nearest to at: near, during, baleen, within, lombardo, waals, via, in,
Nearest to history: list, politics, overview, timeline, lists, geography, demographics, origins,
Average loss at step 572000: 3.009243
Average loss at step 574000: 3.104504
Average loss at step 576000: 3.085287
Average loss at step 578000: 3.063095
Average loss at step 580000: 3.074637
Nearest to this: which, it, another, there, vdp, wilmut, each, some,
Nearest to more: less, fewer, most, quite, very, better, fairly, greater,
Nearest to system: systems, scheme, platform, procedure, network, method, bundesrat, district,
Nearest to his: her, their, my, our, its, your, the, yearning,
Nearest to while: although, whilst, though, when, whereas, furthermore, however, before,
Nearest to after: before, despite, during, when, thereafter, afterwards, while, without,
Nearest to by: through, using, supranationalism, walpole, from, with, against, gladys,
Nearest to has: had, have, is, contains, requires, having, was, holds,
Nearest to six: eight, four, five, seven, nine, three, two, zero,
Nearest to seven: six, five, eight, nine, four, three, zero, two,
Nearest to only: actually, eagerly, nowhere, always, either, best, specifically, correct,
Nearest to were: are, was, have, had, tend, remain, these, those,
Nearest to two: three, four, six, five, one, seven, eight, zero,
Nearest to for: of, including, vigilante, toolkit, against, detonator, aafc, without,
Nearest to at: near, headlined, lombardo, during, baleen, chola, fets, unlucky,
Nearest to history: list, timeline, overview, origins, politics, biography, course, lists,
Average loss at step 582000: 3.081187
Average loss at step 584000: 3.004194
Average loss at step 586000: 3.073145
Average loss at step 588000: 3.092090
Average loss at step 590000: 3.137104
Nearest to this: which, it, another, the, each, instead, that, metadata,
Nearest to more: less, fewer, most, quite, better, worse, larger, very,
Nearest to system: systems, scheme, platform, network, procedure, method, unit, node,
Nearest to his: her, their, my, its, our, your, whose, yearning,
Nearest to while: although, whilst, though, including, when, unlike, whereas, however,
Nearest to after: before, during, despite, when, without, while, thereafter, afterwards,
Nearest to by: through, when, using, including, from, involves, without, via,
Nearest to has: had, have, requires, is, having, contains, maintains, bord,
Nearest to six: five, eight, four, seven, nine, three, two, zero,
Nearest to seven: five, six, eight, four, nine, three, zero, two,
Nearest to only: actually, eagerly, nowhere, cameo, sole, adversely, mid, specifically,
Nearest to were: are, was, tend, had, although, have, remain, these,
Nearest to two: three, four, six, one, seven, five, eight, nine,
Nearest to for: including, of, against, while, filenames, in, during, toolkit,
Nearest to at: in, on, near, headlined, during, baleen, via, beside,
Nearest to history: list, timeline, origins, course, continuation, politics, overview, heritage,
Average loss at step 592000: 3.133770
Average loss at step 594000: 3.115518
Average loss at step 596000: 3.101630
Average loss at step 598000: 3.106099
Average loss at step 600000: 3.125056
Nearest to this: it, which, some, another, any, instead, latter, thanksgiving,
Nearest to more: less, fewer, most, worse, increasingly, quite, better, very,
Nearest to system: systems, network, platform, scheme, unit, model, procedure, district,
Nearest to his: her, their, my, your, its, our, the, playfulness,
Nearest to while: although, whilst, though, when, before, unless, however, after,
Nearest to after: before, during, despite, when, while, without, thereafter, afterwards,
Nearest to by: through, involves, including, without, when, wozniak, from, via,
Nearest to has: had, have, requires, is, was, contains, holds, bord,
Nearest to six: eight, seven, four, five, nine, three, two, zero,
Nearest to seven: five, eight, nine, six, four, three, zero, one,
Nearest to only: actually, eagerly, adversely, jury, just, reciprocated, nowhere, transformer,
Nearest to were: are, was, have, had, these, tend, although, remain,
Nearest to two: three, four, five, one, six, seven, eight, zero,
Nearest to for: including, against, if, without, of, underscore, includes, detonator,
Nearest to at: headlined, baleen, near, tristar, in, via, mv, during,
Nearest to history: list, timeline, origins, continuation, course, heritage, lists, rico,
Average loss at step 602000: 3.007431
Average loss at step 604000: 3.086790
Average loss at step 606000: 2.988670
Average loss at step 608000: 3.099154
Average loss at step 610000: 3.062137
Nearest to this: it, which, another, the, thanksgiving, adventists, metadata, what,
Nearest to more: less, fewer, most, quite, very, greater, increasingly, stronger,
Nearest to system: systems, network, scheme, platform, container, unit, terminology, method,
Nearest to his: her, their, my, your, its, our, whose, the,
Nearest to while: although, whilst, though, when, whereas, involves, unless, after,
Nearest to after: before, during, despite, when, afterwards, without, while, thereafter,
Nearest to by: through, including, gynt, besides, kool, from, provoke, privately,
Nearest to has: had, have, is, maintains, holds, was, having, requires,
Nearest to six: seven, four, eight, nine, five, three, two, zero,
Nearest to seven: six, eight, four, five, nine, three, zero, two,
Nearest to only: eagerly, adversely, jury, actually, reciprocated, sole, correct, transformer,
Nearest to were: are, was, have, remain, these, had, seem, those,
Nearest to two: three, four, five, six, seven, one, eight, zero,
Nearest to for: after, despite, aafc, including, underscore, detonator, unreachable, without,
Nearest to at: in, within, baleen, during, headlined, via, near, unlucky,
Nearest to history: list, timeline, origins, comparison, overview, heritage, course, continuation,
Average loss at step 612000: 3.077700
Average loss at step 614000: 3.132779
Average loss at step 616000: 3.080767
Average loss at step 618000: 2.929647
Average loss at step 620000: 3.090591
Nearest to this: which, it, some, each, the, vdp, instead, another,
Nearest to more: less, quite, fewer, very, increasingly, most, worse, rather,
Nearest to system: systems, network, scheme, platform, program, practice, terminology, suppression,
Nearest to his: her, their, my, its, your, our, the, whose,
Nearest to while: although, whilst, though, when, whereas, however, furthermore, after,
Nearest to after: before, despite, during, when, afterwards, while, without, thereafter,
Nearest to by: through, wozniak, bloch, from, was, sunspot, hazlitt, besides,
Nearest to has: had, have, is, maintains, requires, bord, was, holds,
Nearest to six: seven, four, five, eight, three, nine, two, zero,
Nearest to seven: five, six, eight, nine, four, three, zero, two,
Nearest to only: eagerly, adversely, courtly, reciprocated, approximately, jury, actually, compromised,
Nearest to were: are, was, have, had, remain, seem, although, tend,
Nearest to two: three, four, five, six, one, seven, eight, zero,
Nearest to for: without, aafc, of, consider, in, when, chrominance, including,
Nearest to at: baleen, in, during, waals, fets, headlined, near, via,
Nearest to history: list, timeline, origins, overview, comparison, geography, lists, course,
Average loss at step 622000: 3.045977
Average loss at step 624000: 3.105924
Average loss at step 626000: 3.106538
Average loss at step 628000: 3.069621
Average loss at step 630000: 3.104102
Nearest to this: it, which, instead, some, what, adventists, there, kinetoscope,
Nearest to more: less, fewer, most, very, greater, quite, better, worse,
Nearest to system: systems, network, method, unit, scheme, procedure, platform, circuit,
Nearest to his: her, their, my, its, your, our, whose, yearning,
Nearest to while: although, whilst, though, when, whereas, however, after, unless,
Nearest to after: before, during, despite, when, afterwards, thereafter, while, without,
Nearest to by: through, during, with, using, involves, from, wozniak, mis,
Nearest to has: had, have, maintains, is, requires, bord, provides, contains,
Nearest to six: eight, seven, four, five, nine, three, two, zero,
Nearest to seven: eight, five, six, four, nine, three, zero, two,
Nearest to only: actually, eagerly, adversely, jury, reciprocated, strictly, approximately, until,
Nearest to were: are, was, have, remain, had, those, been, include,
Nearest to two: three, four, five, six, seven, one, eight, nine,
Nearest to for: during, including, against, cotangent, without, unreachable, consider, meagre,
Nearest to at: near, during, in, fets, baleen, under, headlined, within,
Nearest to history: list, timeline, origins, biography, heritage, course, theravada, politics,
Average loss at step 632000: 3.021877
Average loss at step 634000: 2.962395
Average loss at step 636000: 3.091124
Average loss at step 638000: 2.971997
Average loss at step 640000: 3.015104
Nearest to this: which, it, these, adventists, the, some, each, thanksgiving,
Nearest to more: less, fewer, greater, better, quite, very, worse, most,
Nearest to system: systems, network, platform, unit, scheme, document, terminology, procedure,
Nearest to his: her, their, my, your, its, our, the, whose,
Nearest to while: although, whilst, though, when, however, whereas, including, unlike,
Nearest to after: before, despite, during, afterwards, following, thereafter, when, without,
Nearest to by: through, without, including, with, while, against, involves, opposes,
Nearest to has: had, have, maintains, is, was, requires, bord, holds,
Nearest to six: seven, eight, five, three, nine, four, two, zero,
Nearest to seven: six, nine, eight, five, three, four, zero, two,
Nearest to only: actually, adversely, eagerly, jury, strictly, reciprocated, even, either,
Nearest to were: are, was, remain, seem, have, include, tend, had,
Nearest to two: three, four, five, six, seven, eight, one, nine,
Nearest to for: including, against, without, underscore, of, despite, consider, aafc,
Nearest to at: near, in, under, lombardo, on, during, baleen, ophile,
Nearest to history: list, timeline, biography, heritage, origins, overview, glossary, politics,
Average loss at step 642000: 3.011429
Average loss at step 644000: 3.011957
Average loss at step 646000: 3.125506
Average loss at step 648000: 3.071643
Average loss at step 650000: 3.076341
Nearest to this: which, it, some, adventists, there, these, due, wilmut,
Nearest to more: less, fewer, most, worse, greater, quite, very, better,
Nearest to system: systems, network, platform, scheme, terminology, program, steppes, unit,
Nearest to his: her, their, my, its, your, our, whose, the,
Nearest to while: although, whilst, when, though, however, whereas, unlike, including,
Nearest to after: before, during, despite, when, afterwards, while, following, whilst,
Nearest to by: via, through, including, from, besides, during, when, without,
Nearest to has: had, have, is, having, contains, maintains, bord, was,
Nearest to six: seven, eight, five, four, nine, three, zero, two,
Nearest to seven: nine, six, eight, five, four, three, zero, two,
Nearest to only: actually, eagerly, adversely, already, platonist, theoretically, nowhere, specifically,
Nearest to were: are, was, those, remain, seem, have, these, tend,
Nearest to two: three, five, four, six, eight, seven, one, zero,
Nearest to for: against, despite, during, after, in, underscore, aafc, toolkit,
Nearest to at: near, in, lombardo, headlined, on, during, towards, webcam,
Nearest to history: list, timeline, origins, heritage, overview, course, geography, politics,
Average loss at step 652000: 2.839440
Average loss at step 654000: 3.121677
Average loss at step 656000: 3.083440
Average loss at step 658000: 3.100867
Average loss at step 660000: 3.110383
Nearest to this: it, which, the, whatever, each, instead, that, some,
Nearest to more: less, most, very, quite, fewer, better, rather, highly,
Nearest to system: systems, network, scheme, platform, generator, program, unit, terminology,
Nearest to his: her, their, my, your, its, our, whose, s,
Nearest to while: although, whilst, when, though, however, whereas, unless, furthermore,
Nearest to after: before, during, despite, afterwards, thereafter, when, following, while,
Nearest to by: through, when, via, from, during, including, besides, with,
Nearest to has: had, have, is, maintains, contains, having, holds, requires,
Nearest to six: seven, five, eight, four, three, nine, two, zero,
Nearest to seven: six, eight, nine, five, four, three, zero, two,
Nearest to only: eagerly, actually, adversely, best, strictly, reciprocated, nowhere, primarily,
Nearest to were: are, was, remain, seem, have, these, had, tend,
Nearest to two: three, four, five, six, seven, eight, one, nine,
Nearest to for: when, despite, consider, during, in, underscore, against, after,
Nearest to at: near, attended, in, lombardo, during, fets, spacesuit, unlucky,
Nearest to history: list, timeline, heritage, origins, geography, evolution, overview, biography,
Average loss at step 662000: 3.107346
Average loss at step 664000: 3.064070
Average loss at step 666000: 2.960489
Average loss at step 668000: 3.075276
Average loss at step 670000: 3.117075
Nearest to this: which, it, another, whatever, instead, there, that, some,
Nearest to more: less, most, quite, very, fewer, greater, highly, better,
Nearest to system: systems, scheme, platform, unit, network, procedure, format, subsystem,
Nearest to his: her, their, my, your, its, our, whose, the,
Nearest to while: whilst, although, though, whereas, when, additionally, after, before,
Nearest to after: before, during, despite, when, thereafter, afterwards, while, without,
Nearest to by: via, through, without, from, including, when, fillmore, involves,
Nearest to has: had, have, contains, is, maintains, requires, bord, includes,
Nearest to six: eight, five, seven, four, nine, three, zero, two,
Nearest to seven: six, eight, five, nine, four, three, zero, two,
Nearest to only: eagerly, adversely, primarily, strictly, actually, nowhere, theoretically, specifically,
Nearest to were: are, was, remain, seem, have, those, these, tend,
Nearest to two: three, four, five, eight, six, seven, one, nine,
Nearest to for: when, if, after, to, during, with, while, maus,
Nearest to at: near, in, during, lombardo, headlined, waals, goblin, baleen,
Nearest to history: list, timeline, heritage, geography, overview, biography, resources, philosophy,
Average loss at step 672000: 3.064276
Average loss at step 674000: 3.047504
Average loss at step 676000: 3.082921
Average loss at step 678000: 3.086596
Average loss at step 680000: 3.091026
Nearest to this: which, it, another, each, ample, vdp, qibla, larps,
Nearest to more: less, quite, most, very, fewer, worse, fairly, greater,
Nearest to system: systems, scheme, platform, unit, network, controllers, steppes, format,
Nearest to his: her, their, its, your, my, our, the, whose,
Nearest to while: whilst, although, though, whereas, after, when, despite, before,
Nearest to after: before, during, despite, while, when, thereafter, afterwards, whilst,
Nearest to by: through, involves, via, including, fillmore, besides, from, wt,
Nearest to has: had, have, is, was, contains, bord, having, provides,
Nearest to six: eight, seven, five, four, three, nine, two, zero,
Nearest to seven: eight, six, nine, five, four, three, zero, two,
Nearest to only: adversely, eagerly, either, primarily, actually, nowhere, strictly, mourns,
Nearest to were: are, was, have, seem, had, lindo, those, including,
Nearest to two: three, four, six, five, eight, seven, one, nine,
Nearest to for: after, of, against, through, despite, and, by, involves,
Nearest to at: near, in, during, on, headlined, waals, goblin, fets,
Nearest to history: list, timeline, geography, overview, origins, scorpions, evolution, northern,
Average loss at step 682000: 3.089974
Average loss at step 684000: 3.018919
Average loss at step 686000: 3.002816
Average loss at step 688000: 3.083558
Average loss at step 690000: 3.041877
Nearest to this: which, it, these, the, some, another, each, there,
Nearest to more: less, most, fewer, greater, quite, smaller, very, fairly,
Nearest to system: systems, scheme, network, method, platform, bundesrat, process, subsystem,
Nearest to his: her, their, its, your, my, our, whose, patroclus,
Nearest to while: whilst, although, though, whereas, however, when, including, moreover,
Nearest to after: before, during, despite, when, afterwards, thereafter, without, while,
Nearest to by: through, including, via, from, fillmore, without, involves, in,
Nearest to has: had, have, is, was, maintains, provides, requires, having,
Nearest to six: five, four, seven, nine, eight, zero, three, two,
Nearest to seven: five, nine, four, three, eight, six, zero, two,
Nearest to only: eagerly, adversely, eurozone, jury, primarily, grotto, theoretically, actually,
Nearest to were: are, was, these, those, have, include, tend, seem,
Nearest to two: three, four, five, seven, six, one, eight, zero,
Nearest to for: cotangent, without, consider, despite, alphorn, after, collation, toolkit,
Nearest to at: near, in, during, fets, baleen, webcam, subsets, tristar,
Nearest to history: list, timeline, overview, geography, heritage, origins, evolution, biography,
Average loss at step 692000: 3.064045
Average loss at step 694000: 3.057010
Average loss at step 696000: 3.074901
Average loss at step 698000: 2.821285
Average loss at step 700000: 2.550709
Nearest to this: which, it, another, adventists, some, thanksgiving, physicalism, his,
Nearest to more: less, fewer, most, quite, greater, worse, very, sirhan,
Nearest to system: systems, scheme, method, network, unit, platform, subsystem, schemes,
Nearest to his: her, their, my, your, its, the, our, s,
Nearest to while: whilst, although, though, when, whereas, however, after, before,
Nearest to after: before, during, despite, thereafter, when, while, without, afterwards,
Nearest to by: via, including, with, from, when, for, through, under,
Nearest to has: had, have, is, maintains, was, holds, bord, requires,
Nearest to six: seven, eight, nine, five, four, three, zero, two,
Nearest to seven: six, five, eight, nine, four, three, zero, two,
Nearest to only: eagerly, adversely, eurozone, platonist, mourns, egyptians, reciprocated, primarily,
Nearest to were: are, was, these, had, remain, thatched, seem, been,
Nearest to two: three, five, four, six, seven, eight, one, zero,
Nearest to for: by, after, of, during, prestigious, mclachlan, djamena, cotangent,
Nearest to at: near, in, during, after, attended, on, spacesuit, licking,
Nearest to history: list, timeline, heritage, tradition, biography, overview, geography, course,
Average loss at step 702000: 2.825185
Average loss at step 704000: 2.809209
Average loss at step 706000: 2.982247
Average loss at step 708000: 3.029927
Average loss at step 710000: 3.051449
Nearest to this: which, it, another, his, some, physicalism, the, adventists,
Nearest to more: less, most, fewer, quite, greater, better, worse, very,
Nearest to system: systems, scheme, method, procedure, unit, discriminant, terminology, practice,
Nearest to his: their, her, my, its, your, our, whose, the,
Nearest to while: whilst, although, though, when, whereas, after, before, however,
Nearest to after: before, despite, without, when, thereafter, during, afterwards, while,
Nearest to by: through, without, with, involves, featuring, walther, for, syme,
Nearest to has: had, have, is, holds, maintains, was, requires, offers,
Nearest to six: seven, eight, four, five, nine, three, zero, two,
Nearest to seven: six, eight, four, five, nine, three, zero, one,
Nearest to only: eagerly, adversely, eurozone, mourns, jury, either, last, reciprocated,
Nearest to were: are, was, these, have, remain, include, tend, had,
Nearest to two: three, four, five, six, one, seven, eight, zero,
Nearest to for: without, despite, of, after, by, and, prestigious, aafc,
Nearest to at: near, during, in, attended, goblin, on, despite, headlined,
Nearest to history: list, tradition, timeline, heritage, origins, biography, archaeology, geography,
Average loss at step 712000: 2.972750
Average loss at step 714000: 3.066677
Average loss at step 716000: 3.098670
Average loss at step 718000: 3.067005
Average loss at step 720000: 3.098832
Nearest to this: which, it, any, some, another, that, whatever, physicalism,
Nearest to more: less, most, fewer, quite, greater, better, worse, increasingly,
Nearest to system: systems, scheme, network, platform, method, procedure, terminology, subsystem,
Nearest to his: her, their, my, your, its, our, whose, the,
Nearest to while: whilst, although, when, though, whereas, however, after, before,
Nearest to after: before, despite, during, thereafter, afterwards, when, while, without,
Nearest to by: through, via, wozniak, with, without, including, gladys, besides,
Nearest to has: had, have, is, holds, maintains, requires, was, contains,
Nearest to six: seven, four, five, eight, nine, three, zero, two,
Nearest to seven: eight, six, four, nine, five, three, zero, two,
Nearest to only: eagerly, adversely, actually, eurozone, mourns, strictly, primarily, nowhere,
Nearest to were: are, was, had, have, those, these, been, lindo,
Nearest to two: three, four, five, six, one, eight, seven, zero,
Nearest to for: without, against, consider, despite, genocidal, paulette, kumquats, cotangent,
Nearest to at: near, during, goblin, webcam, on, tristar, in, adair,
Nearest to history: list, heritage, timeline, tradition, origins, archaeology, scorpions, course,
Average loss at step 722000: 2.898756
Average loss at step 724000: 3.003203
Average loss at step 726000: 3.084940
Average loss at step 728000: 2.992725
Average loss at step 730000: 3.102736
Nearest to this: which, it, another, whatever, some, any, wilmut, adventists,
Nearest to more: less, most, fewer, quite, faster, greater, fairly, smaller,
Nearest to system: systems, scheme, network, method, platform, gui, subsystem, procedure,
Nearest to his: her, their, my, your, its, our, the, dividends,
Nearest to while: whilst, although, whereas, though, when, after, however, moreover,
Nearest to after: before, during, despite, when, while, afterwards, without, thereafter,
Nearest to by: through, wozniak, via, from, without, with, involves, walther,
Nearest to has: had, have, is, requires, contains, holds, maintains, bord,
Nearest to six: four, seven, eight, five, three, nine, two, zero,
Nearest to seven: four, six, eight, five, three, nine, zero, two,
Nearest to only: eagerly, adversely, eurozone, nowhere, mourns, actually, either, jury,
Nearest to were: are, those, these, was, include, seem, tend, remain,
Nearest to two: three, four, five, six, seven, eight, zero, one,
Nearest to for: toolkit, without, against, disposable, prevents, kumquats, by, surgical,
Nearest to at: near, during, on, within, in, webcam, around, goblin,
Nearest to history: origins, timeline, list, heritage, tradition, archaeology, overview, course,
Average loss at step 732000: 3.040887
Average loss at step 734000: 3.123239
Average loss at step 736000: 3.048233
Average loss at step 738000: 2.963548
Average loss at step 740000: 2.752446
Nearest to this: which, it, another, the, that, some, fledging, any,
Nearest to more: less, fewer, greater, most, quite, worse, faster, smaller,
Nearest to system: systems, scheme, network, platform, schemes, infrastructure, district, practice,
Nearest to his: her, their, its, my, your, our, whose, jointly,
Nearest to while: although, whilst, though, whereas, moreover, after, however, when,
Nearest to after: before, during, despite, while, following, when, afterwards, without,
Nearest to by: through, via, wozniak, from, viaduct, in, with, fillmore,
Nearest to has: had, have, is, was, having, maintains, since, bord,
Nearest to six: eight, four, five, seven, three, nine, two, zero,
Nearest to seven: four, eight, six, three, five, nine, zero, two,
Nearest to only: eurozone, sole, adversely, eagerly, courtly, cmyk, smallest, anglicanism,
Nearest to were: are, was, have, although, those, had, these, tend,
Nearest to two: three, four, one, five, six, seven, eight, zero,
Nearest to for: despite, against, after, of, underscore, prevents, djamena, tba,
Nearest to at: during, near, within, despite, baleen, in, around, goblin,
Nearest to history: list, timeline, overview, origins, politics, geography, archaeology, demographics,
Average loss at step 742000: 2.999408
Average loss at step 744000: 3.037118
Average loss at step 746000: 2.861588
Average loss at step 748000: 3.043885
Average loss at step 750000: 3.131837
Nearest to this: which, it, that, some, the, adventists, another, what,
Nearest to more: less, fewer, greater, most, quite, better, faster, smaller,
Nearest to system: systems, network, scheme, platform, method, microprocessor, process, procedure,
Nearest to his: her, their, your, my, its, our, jointly, innocuous,
Nearest to while: whilst, although, when, after, though, whereas, however, moreover,
Nearest to after: before, while, despite, during, afterwards, when, thereafter, without,
Nearest to by: through, via, numan, wozniak, including, aldrich, lyons, featuring,
Nearest to has: had, have, is, having, contains, maintains, requires, produces,
Nearest to six: four, eight, seven, five, nine, three, zero, two,
Nearest to seven: eight, four, six, five, nine, three, zero, two,
Nearest to only: adversely, actually, eagerly, nowhere, specifically, necessarily, cmyk, eurozone,
Nearest to were: are, was, those, have, had, tend, while, bunge,
Nearest to two: three, four, five, six, one, eight, seven, zero,
Nearest to for: in, against, if, toolkit, while, after, underscore, without,
Nearest to at: in, during, near, within, goblin, around, on, faceless,
Nearest to history: timeline, list, archaeology, tradition, overview, heritage, career, origins,
Average loss at step 752000: 3.024566
Average loss at step 754000: 2.950487
Average loss at step 756000: 3.028354
Average loss at step 758000: 3.074039
Average loss at step 760000: 3.120495
Nearest to this: it, which, some, qibla, thanksgiving, another, adventists, kinetoscope,
Nearest to more: less, fewer, most, quite, greater, worse, better, smaller,
Nearest to system: systems, platform, scheme, network, process, method, procedure, istituto,
Nearest to his: her, their, your, its, my, our, the, innocuous,
Nearest to while: whilst, although, when, whereas, though, after, moreover, however,
Nearest to after: before, during, when, despite, while, afterwards, without, thereafter,
Nearest to by: through, via, including, wozniak, without, during, when, for,
Nearest to has: had, have, is, requires, maintains, contains, having, was,
Nearest to six: eight, four, seven, five, nine, three, two, zero,
Nearest to seven: five, four, six, eight, nine, three, zero, two,
Nearest to only: actually, adversely, either, eagerly, sole, necessarily, specifically, cmyk,
Nearest to were: are, those, was, tend, seem, have, had, remain,
Nearest to two: three, four, five, seven, six, eight, one, nine,
Nearest to for: including, during, in, without, mclachlan, of, underscore, against,
Nearest to at: near, in, during, within, attended, chola, goblin, viewable,
Nearest to history: timeline, list, overview, sel, origins, scorpions, tradition, geography,
Average loss at step 762000: 3.090747
Average loss at step 764000: 3.058238
Average loss at step 766000: 2.995850
Average loss at step 768000: 3.081376
Average loss at step 770000: 2.969790
Nearest to this: it, which, any, the, another, thanksgiving, what, whatever,
Nearest to more: less, fewer, most, quite, worse, greater, stronger, better,
Nearest to system: systems, scheme, procedure, practice, bundesrat, method, grouping, platform,
Nearest to his: her, their, my, your, its, our, personal, the,
Nearest to while: whilst, although, when, though, whereas, moreover, however, after,
Nearest to after: before, during, despite, when, while, afterwards, without, following,
Nearest to by: through, including, without, during, with, wozniak, from, provoke,
Nearest to has: had, have, is, requires, contains, having, maintains, was,
Nearest to six: seven, eight, four, nine, five, three, zero, two,
Nearest to seven: eight, six, nine, four, five, three, zero, two,
Nearest to only: actually, adversely, eagerly, necessarily, always, either, sole, transformer,
Nearest to were: are, was, remain, those, had, although, tend, seem,
Nearest to two: three, four, six, five, seven, eight, nine, one,
Nearest to for: against, after, without, in, despite, including, during, aafc,
Nearest to at: near, during, in, milken, ih, viewable, tristar, within,
Nearest to history: list, timeline, overview, origins, sel, geography, politics, superfluid,
Average loss at step 772000: 3.048612
Average loss at step 774000: 3.026529
Average loss at step 776000: 2.949433
Average loss at step 778000: 2.931646
Average loss at step 780000: 2.991305
Nearest to this: which, it, another, any, thanksgiving, ample, what, unsurprising,
Nearest to more: less, fewer, quite, most, very, greater, fairly, worse,
Nearest to system: systems, scheme, method, network, platform, procedure, polyhedron, grouping,
Nearest to his: her, their, my, your, its, our, personal, whose,
Nearest to while: whilst, although, though, whereas, when, moreover, after, despite,
Nearest to after: before, during, despite, when, afterwards, without, while, thereafter,
Nearest to by: through, from, without, with, during, against, via, when,
Nearest to has: had, have, is, was, having, requires, contains, maintains,
Nearest to six: eight, seven, five, four, nine, three, zero, two,
Nearest to seven: eight, six, five, four, nine, three, zero, two,
Nearest to only: eagerly, adversely, sole, mourns, actually, canteen, fda, until,
Nearest to were: are, was, remain, had, those, seem, have, zealous,
Nearest to two: three, four, five, six, seven, eight, nine, one,
Nearest to for: without, including, chrominance, in, despite, after, of, against,
Nearest to at: in, near, during, within, goblin, fets, attended, under,
Nearest to history: origins, list, timeline, archaeology, tradition, politics, heritage, overview,
Average loss at step 782000: 3.061276
Average loss at step 784000: 3.022179
Average loss at step 786000: 3.005363
Average loss at step 788000: 3.041366
Average loss at step 790000: 3.043828
Nearest to this: which, it, what, another, thanksgiving, the, instead, framing,
Nearest to more: less, fewer, most, very, greater, quite, fairly, worse,
Nearest to system: systems, scheme, network, method, procedure, platform, cisc, controllers,
Nearest to his: her, their, my, your, its, our, whose, jointly,
Nearest to while: whilst, although, when, though, whereas, after, despite, but,
Nearest to after: before, despite, during, when, afterwards, while, thereafter, without,
Nearest to by: through, via, from, without, under, with, when, against,
Nearest to has: had, have, is, having, requires, contains, maintains, was,
Nearest to six: four, eight, seven, five, three, nine, zero, two,
Nearest to seven: eight, nine, five, six, four, three, zero, two,
Nearest to only: adversely, eagerly, fda, sole, jury, nowhere, eurozone, henze,
Nearest to were: are, was, those, have, had, these, seem, contain,
Nearest to two: three, four, six, five, eight, seven, one, nine,
Nearest to for: including, in, of, after, overwriting, when, without, meagre,
Nearest to at: near, in, during, within, gernika, goblin, viewable, under,
Nearest to history: timeline, list, origins, overview, archaeology, continuation, heritage, lists,
Average loss at step 792000: 3.070572
Average loss at step 794000: 3.056938
Average loss at step 796000: 3.056628
Average loss at step 798000: 2.923739
Average loss at step 800000: 2.971241
Nearest to this: which, it, thanksgiving, some, that, wtoo, another, vangelis,
Nearest to more: less, fewer, most, very, greater, fairly, increasingly, smaller,
Nearest to system: systems, network, scheme, suppression, platform, mechanisms, procedure, schemes,
Nearest to his: her, their, my, your, its, our, the, s,
Nearest to while: whilst, although, though, when, however, whereas, including, began,
Nearest to after: before, despite, during, when, afterwards, without, while, thereafter,
Nearest to by: through, via, with, using, when, theorist, cavett, against,
Nearest to has: had, have, is, contains, maintains, having, offers, was,
Nearest to six: four, eight, five, seven, three, nine, two, zero,
Nearest to seven: eight, five, six, four, nine, three, two, zero,
Nearest to only: adversely, eagerly, platonist, specifically, even, transformer, fda, henze,
Nearest to were: are, was, have, tend, had, remain, been, these,
Nearest to two: three, four, six, five, seven, one, eight, nine,
Nearest to for: including, purpurea, rosso, sparrows, against, lal, cert, underscore,
Nearest to at: near, in, during, gernika, within, baleen, milken, via,
Nearest to history: list, timeline, lists, origins, geography, scorpions, overview, biography,
Average loss at step 802000: 3.060134
Average loss at step 804000: 3.065723
Average loss at step 806000: 3.065442
Average loss at step 808000: 3.057130
Average loss at step 810000: 3.030990
Nearest to this: which, it, some, there, another, telephus, that, instead,
Nearest to more: less, fewer, most, fairly, quite, greater, very, highly,
Nearest to system: systems, scheme, network, method, procedure, excelling, platform, container,
Nearest to his: her, their, my, your, its, our, the, s,
Nearest to while: whilst, although, when, though, however, whereas, despite, involves,
Nearest to after: before, despite, during, when, without, afterwards, thereafter, while,
Nearest to by: through, with, against, involves, via, without, after, when,
Nearest to has: had, have, is, having, contains, holds, was, maintains,
Nearest to six: eight, four, five, seven, three, nine, two, zero,
Nearest to seven: eight, six, five, four, nine, three, zero, two,
Nearest to only: eagerly, either, adversely, specifically, actually, even, best, anglicanism,
Nearest to were: are, was, have, remain, tend, had, became, although,
Nearest to two: three, four, six, five, one, eight, seven, nine,
Nearest to for: in, against, of, during, lal, including, despite, after,
Nearest to at: near, in, on, during, headlined, unlucky, webcam, within,
Nearest to history: list, timeline, lists, origins, career, overview, premier, phonology,
Average loss at step 812000: 3.005979
Average loss at step 814000: 3.085745
Average loss at step 816000: 3.071066
Average loss at step 818000: 3.077893
Average loss at step 820000: 3.034514
Nearest to this: which, it, there, some, that, coffers, the, substandard,
Nearest to more: less, fewer, most, increasingly, quite, greater, fairly, worse,
Nearest to system: systems, scheme, network, method, procedure, platform, bundesrat, metrodome,
Nearest to his: her, their, my, its, your, our, yearning, dividends,
Nearest to while: whilst, although, when, though, whereas, after, furthermore, however,
Nearest to after: before, despite, during, when, while, without, afterwards, whilst,
Nearest to by: through, including, via, against, wahhabis, without, involves, with,
Nearest to has: had, have, contains, having, is, requires, maintains, was,
Nearest to six: four, eight, five, seven, three, nine, two, zero,
Nearest to seven: eight, five, six, nine, four, three, zero, two,
Nearest to only: adversely, eagerly, specifically, either, sole, actually, strictly, least,
Nearest to were: are, was, have, tend, those, been, seem, remain,
Nearest to two: three, four, six, five, seven, eight, one, nine,
Nearest to for: including, cotangent, without, against, despite, chrominance, mclachlan, of,
Nearest to at: unlucky, webcam, fets, headlined, near, toward, lombardo, within,
Nearest to history: list, lists, timeline, overview, origins, phonology, archaeology, geography,
Average loss at step 822000: 3.019078
Average loss at step 824000: 2.795178
Average loss at step 826000: 2.988127
Average loss at step 828000: 3.062576
Average loss at step 830000: 3.039546
Nearest to this: which, it, some, the, there, another, each, vdp,
Nearest to more: less, fewer, most, greater, increasingly, better, stronger, worse,
Nearest to system: systems, scheme, network, procedure, bundesrat, method, platform, polyhedron,
Nearest to his: her, their, my, your, our, its, the, yearning,
Nearest to while: whilst, although, whereas, though, when, furthermore, however, including,
Nearest to after: before, despite, during, when, without, while, whilst, afterwards,
Nearest to by: through, including, from, without, with, via, levin, involves,
Nearest to has: had, have, is, contains, maintains, was, requires, holds,
Nearest to six: eight, five, four, seven, nine, three, two, zero,
Nearest to seven: six, five, eight, nine, four, three, two, zero,
Nearest to only: eagerly, sole, either, adversely, specifically, actually, best, nowhere,
Nearest to were: are, was, have, tend, seem, been, remain, those,
Nearest to two: three, six, four, five, seven, eight, one, nine,
Nearest to for: including, of, where, against, after, in, despite, choreography,
Nearest to at: near, in, during, lombardo, baleen, toward, within, unlucky,
Nearest to history: list, timeline, overview, lists, origins, geography, resources, politics,
Average loss at step 832000: 3.038058
Average loss at step 834000: 3.034417
Average loss at step 836000: 3.050058
Average loss at step 838000: 2.919158
Average loss at step 840000: 3.044323
Nearest to this: which, it, what, another, wtoo, instead, wilmut, qibla,
Nearest to more: less, fewer, stronger, most, quite, worse, very, better,
Nearest to system: systems, scheme, network, method, procedure, platform, methods, terminology,
Nearest to his: her, their, my, your, its, our, innocuous, courage,
Nearest to while: whilst, although, when, though, whereas, after, including, unless,
Nearest to after: before, despite, during, when, while, thereafter, afterwards, without,
Nearest to by: through, including, from, when, levin, involves, fantastical, bergman,
Nearest to has: had, have, is, requires, maintains, contains, was, having,
Nearest to six: five, eight, four, seven, three, nine, two, zero,
Nearest to seven: six, eight, five, nine, four, three, zero, two,
Nearest to only: sole, either, nowhere, actually, eagerly, best, adversely, specifically,
Nearest to were: are, was, have, those, had, tend, became, remain,
Nearest to two: three, six, four, five, one, seven, eight, zero,
Nearest to for: against, including, after, when, before, during, in, from,
Nearest to at: during, gernika, near, tristar, headlined, in, unlucky, coincidentally,
Nearest to history: timeline, list, origins, lists, overview, biography, geography, continuation,
Average loss at step 842000: 3.065170
Average loss at step 844000: 3.091487
Average loss at step 846000: 3.096404
Average loss at step 848000: 3.085451
Average loss at step 850000: 3.060037
Nearest to this: which, it, the, another, some, each, any, wilmut,
Nearest to more: less, fewer, quite, most, stronger, very, fairly, worse,
Nearest to system: systems, scheme, network, platform, methods, bundesrat, procedure, dewey,
Nearest to his: her, their, my, your, our, its, the, whose,
Nearest to while: whilst, although, when, though, after, unless, whereas, unlike,
Nearest to after: before, despite, during, when, while, without, although, thereafter,
Nearest to by: through, when, without, levin, from, via, including, involves,
Nearest to has: had, have, is, having, requires, maintains, was, contains,
Nearest to six: eight, seven, five, four, nine, three, two, zero,
Nearest to seven: five, eight, six, four, nine, three, zero, two,
Nearest to only: eagerly, actually, either, specifically, adversely, sole, jury, nowhere,
Nearest to were: are, was, have, had, those, tend, although, these,
Nearest to two: three, four, six, five, seven, one, eight, nine,
Nearest to for: including, against, of, after, when, without, includes, despite,
Nearest to at: baleen, during, in, headlined, graduate, finned, via, unlucky,
Nearest to history: list, timeline, origins, lists, politics, continuation, theravada, selection,
Average loss at step 852000: 3.060712
Average loss at step 854000: 3.057596
Average loss at step 856000: 2.978574
Average loss at step 858000: 3.065821
Average loss at step 860000: 2.942591
Nearest to this: it, which, another, some, any, there, the, what,
Nearest to more: less, fewer, most, stronger, quite, very, extremely, worse,
Nearest to system: systems, scheme, network, polyhedron, bundesrat, method, subsystem, gui,
Nearest to his: her, their, my, your, our, its, the, yearning,
Nearest to while: whilst, although, when, though, whereas, involves, unless, additionally,
Nearest to after: before, despite, during, when, afterwards, thereafter, without, while,
Nearest to by: through, when, including, involves, without, from, via, with,
Nearest to has: had, have, is, maintains, was, having, supports, holds,
Nearest to six: seven, five, four, eight, three, nine, two, zero,
Nearest to seven: six, eight, nine, five, four, three, zero, two,
Nearest to only: adversely, eagerly, even, transformer, actually, reciprocated, sole, either,
Nearest to were: are, was, have, seem, these, been, had, those,
Nearest to two: three, four, six, five, seven, one, eight, zero,
Nearest to for: of, including, in, with, chrominance, when, cotangent, after,
Nearest to at: baleen, tristar, during, near, on, headlined, in, from,
Nearest to history: list, timeline, origins, selection, geography, lists, overview, continuation,
Average loss at step 862000: 3.069182
Average loss at step 864000: 3.032580
Average loss at step 866000: 3.023311
Average loss at step 868000: 3.104369
Average loss at step 870000: 3.045328
Nearest to this: it, which, some, another, each, there, what, any,
Nearest to more: less, fewer, very, most, stronger, quite, extremely, increasingly,
Nearest to system: systems, network, scheme, program, subsystem, platform, process, collapse,
Nearest to his: her, their, my, its, your, our, the, courage,
Nearest to while: whilst, although, though, when, whereas, however, involves, but,
Nearest to after: before, during, despite, when, afterwards, without, while, thereafter,
Nearest to by: through, including, via, besides, from, wozniak, hazlitt, levin,
Nearest to has: had, have, is, maintains, holds, bord, offers, requires,
Nearest to six: seven, four, five, eight, nine, three, zero, two,
Nearest to seven: six, eight, five, four, nine, three, zero, two,
Nearest to only: adversely, eagerly, reciprocated, even, sole, courtly, actually, jury,
Nearest to were: are, was, have, had, remain, seem, although, those,
Nearest to two: three, four, five, six, one, seven, eight, zero,
Nearest to for: including, in, collation, when, unreachable, interdiction, aafc, without,
Nearest to at: during, near, in, baleen, tristar, within, headlined, beside,
Nearest to history: list, timeline, origins, lists, overview, selection, continuation, resources,
Average loss at step 872000: 2.876581
Average loss at step 874000: 3.070696
Average loss at step 876000: 3.011452
Average loss at step 878000: 3.069553
Average loss at step 880000: 3.064873
Nearest to this: which, it, the, each, some, sublimation, any, vdp,
Nearest to more: less, fewer, most, quite, stronger, very, extremely, fairly,
Nearest to system: systems, network, scheme, terminology, method, container, subsystem, mechanisms,
Nearest to his: her, their, my, its, your, our, whose, s,
Nearest to while: whilst, although, when, though, whereas, after, stopped, involves,
Nearest to after: before, during, despite, when, afterwards, thereafter, while, without,
Nearest to by: from, through, via, including, with, hazlitt, for, in,
Nearest to has: had, have, is, maintains, was, requires, having, seeks,
Nearest to six: seven, eight, five, four, nine, three, zero, two,
Nearest to seven: six, five, eight, nine, four, three, zero, one,
Nearest to only: adversely, actually, eagerly, jury, primarily, reciprocated, sole, transformer,
Nearest to were: are, was, have, seem, remain, had, comprise, those,
Nearest to two: three, four, six, one, five, eight, seven, nine,
Nearest to for: without, aafc, by, krantz, of, including, from, in,
Nearest to at: near, baleen, in, headlined, during, tristar, waals, mv,
Nearest to history: timeline, list, origins, geography, biography, continuation, glossary, lists,
Average loss at step 882000: 3.037371
Average loss at step 884000: 3.025110
Average loss at step 886000: 3.016978
Average loss at step 888000: 2.922993
Average loss at step 890000: 3.057451
Nearest to this: which, it, the, some, another, each, any, these,
Nearest to more: less, fewer, most, stronger, greater, better, worse, quite,
Nearest to system: systems, network, scheme, program, platform, container, generator, circuit,
Nearest to his: her, their, my, your, its, our, the, whose,
Nearest to while: whilst, although, though, however, whereas, when, including, additionally,
Nearest to after: before, despite, during, when, afterwards, thereafter, without, whilst,
Nearest to by: including, through, with, beattie, involves, via, without, levin,
Nearest to has: had, have, maintains, having, requires, is, offers, bord,
Nearest to six: five, seven, eight, four, nine, three, zero, two,
Nearest to seven: six, five, eight, nine, four, three, zero, two,
Nearest to only: actually, adversely, jury, either, eagerly, strictly, sole, primarily,
Nearest to were: are, was, have, include, seem, remain, been, had,
Nearest to two: three, four, six, one, five, seven, eight, nine,
Nearest to for: including, without, against, in, of, by, from, cotangent,
Nearest to at: near, in, baleen, during, within, lombardo, finned, toward,
Nearest to history: list, timeline, biography, lists, origins, glossary, politics, overview,
Average loss at step 892000: 2.927941
Average loss at step 894000: 2.996813
Average loss at step 896000: 2.940803
Average loss at step 898000: 2.993709
Average loss at step 900000: 3.082123
Nearest to this: which, it, some, these, the, unsurprising, another, similar,
Nearest to more: less, fewer, most, stronger, greater, quite, better, worse,
Nearest to system: systems, network, scheme, generator, platform, facility, program, controllers,
Nearest to his: her, their, my, its, your, our, the, whose,
Nearest to while: whilst, although, when, though, whereas, however, unlike, besides,
Nearest to after: before, despite, during, when, afterwards, without, while, thereafter,
Nearest to by: without, when, through, via, involves, including, hazlitt, provoke,
Nearest to has: had, have, maintains, requires, is, holds, having, offers,
Nearest to six: seven, eight, five, three, nine, four, two, zero,
Nearest to seven: six, nine, five, eight, three, four, zero, two,
Nearest to only: actually, even, eagerly, either, adversely, jury, objectors, reciprocated,
Nearest to were: are, was, seem, remain, those, had, have, tend,
Nearest to two: three, four, five, six, one, seven, eight, zero,
Nearest to for: despite, against, without, after, including, of, teenaged, interdiction,
Nearest to at: near, in, lombardo, towards, waals, via, during, attended,
Nearest to history: list, origins, timeline, glossary, overview, biography, heritage, politics,
Average loss at step 902000: 3.026955
Average loss at step 904000: 3.044028
Average loss at step 906000: 2.829045
Average loss at step 908000: 3.084989
Average loss at step 910000: 3.043045
Nearest to this: which, it, the, some, another, these, instead, conclusively,
Nearest to more: less, fewer, most, quite, better, stronger, very, greater,
Nearest to system: systems, network, scheme, framework, platform, generator, password, model,
Nearest to his: her, their, my, your, its, our, the, whose,
Nearest to while: whilst, although, when, though, whereas, furthermore, however, despite,
Nearest to after: before, during, when, despite, afterwards, following, thereafter, while,
Nearest to by: through, from, via, when, during, denigrating, levin, in,
Nearest to has: had, have, is, contains, was, maintains, since, holds,
Nearest to six: eight, five, seven, four, nine, three, zero, two,
Nearest to seven: six, five, nine, eight, four, three, zero, two,
Nearest to only: actually, eagerly, adversely, reciprocated, primarily, objectors, platonist, best,
Nearest to were: are, seem, was, these, remain, those, have, tend,
Nearest to two: three, five, four, six, seven, one, eight, nine,
Nearest to for: after, when, of, in, against, despite, analyzes, during,
Nearest to at: in, near, on, lombardo, spacesuit, toward, during, chola,
Nearest to history: list, timeline, geography, overview, origins, glossary, demographics, ecology,
Average loss at step 912000: 3.058595
Average loss at step 914000: 3.073004
Average loss at step 916000: 3.070257
Average loss at step 918000: 3.021263
Average loss at step 920000: 2.931926
Nearest to this: which, it, some, another, that, these, the, instead,
Nearest to more: less, most, quite, very, fewer, greater, stronger, better,
Nearest to system: systems, scheme, steppes, network, bundesrat, reform, platform, mainstay,
Nearest to his: her, their, my, your, its, whose, our, the,
Nearest to while: whilst, although, though, when, whereas, besides, after, despite,
Nearest to after: before, during, when, despite, thereafter, while, afterwards, following,
Nearest to by: through, via, during, when, against, from, with, denigrating,
Nearest to has: had, have, is, was, having, maintains, bord, supports,
Nearest to six: four, five, eight, seven, three, nine, two, zero,
Nearest to seven: six, five, eight, four, nine, three, zero, two,
Nearest to only: eagerly, actually, adversely, primarily, reciprocated, strictly, even, platonist,
Nearest to were: are, was, remain, have, those, seem, include, been,
Nearest to two: three, four, five, six, seven, eight, one, nine,
Nearest to for: after, while, mclachlan, chrominance, during, when, hopwood, although,
Nearest to at: near, during, baleen, lombardo, in, waals, sturgis, attended,
Nearest to history: list, timeline, geography, glossary, overview, heritage, politics, biography,
Average loss at step 922000: 3.045501
Average loss at step 924000: 3.060776
Average loss at step 926000: 3.043445
Average loss at step 928000: 3.011155
Average loss at step 930000: 3.053463
Nearest to this: which, it, another, each, whatever, similar, cannibalism, the,
Nearest to more: less, quite, most, fewer, very, smaller, greater, stronger,
Nearest to system: systems, scheme, network, generator, platform, subsystem, gui, potion,
Nearest to his: her, their, your, my, its, our, whose, the,
Nearest to while: whilst, although, though, whereas, when, but, however, after,
Nearest to after: before, during, despite, when, afterwards, thereafter, while, following,
Nearest to by: through, via, including, involves, benchers, in, levin, fillmore,
Nearest to has: had, have, is, was, maintains, offers, seeks, contains,
Nearest to six: eight, seven, five, four, nine, three, two, zero,
Nearest to seven: six, eight, five, nine, four, three, zero, two,
Nearest to only: eagerly, adversely, primarily, sole, either, strictly, hypnotized, even,
Nearest to were: are, seem, was, have, remain, had, include, those,
Nearest to two: three, four, five, six, seven, eight, one, nine,
Nearest to for: against, after, despite, chrominance, when, during, where, in,
Nearest to at: near, in, beside, headlined, waals, during, on, attended,
Nearest to history: list, timeline, geography, overview, politics, phonology, heritage, origins,
Average loss at step 932000: 3.029275
Average loss at step 934000: 3.068160
Average loss at step 936000: 3.044683
Average loss at step 938000: 2.948530
Average loss at step 940000: 3.018526
Nearest to this: which, it, another, some, each, these, any, similar,
Nearest to more: less, fewer, quite, very, greater, most, stronger, better,
Nearest to system: systems, scheme, network, steppes, bundesrat, potion, subsystem, platform,
Nearest to his: her, their, its, your, my, our, the, whose,
Nearest to while: whilst, although, whereas, though, when, moreover, besides, however,
Nearest to after: before, during, despite, when, afterwards, thereafter, while, following,
Nearest to by: through, via, involves, without, from, wozniak, fillmore, including,
Nearest to has: had, have, is, seeks, contains, was, encompasses, bord,
Nearest to six: seven, five, eight, four, three, nine, zero, two,
Nearest to seven: six, five, nine, four, eight, three, zero, two,
Nearest to only: eagerly, adversely, primarily, actually, grotto, legally, theoretically, specifically,
Nearest to were: are, was, have, had, those, seem, these, remain,
Nearest to two: three, four, seven, five, eight, six, one, zero,
Nearest to for: alphorn, cotangent, baud, of, filenames, maus, includes, analyzes,
Nearest to at: near, in, waals, during, baleen, gambia, comically, primeval,
Nearest to history: list, timeline, overview, geography, origins, archaeology, scorpions, politics,
Average loss at step 942000: 3.048163
Average loss at step 944000: 2.995146
Average loss at step 946000: 3.032246
Average loss at step 948000: 3.012961
Average loss at step 950000: 3.051156
Nearest to this: which, it, another, some, each, cannibalism, his, these,
Nearest to more: less, most, fewer, quite, very, greater, better, fairly,
Nearest to system: systems, scheme, network, method, bundesrat, process, potion, terminology,
Nearest to his: her, their, its, your, my, the, s, our,
Nearest to while: whilst, although, though, whereas, however, when, but, besides,
Nearest to after: before, during, afterwards, when, despite, thereafter, while, without,
Nearest to by: from, through, via, including, with, without, levin, wozniak,
Nearest to has: had, have, is, maintains, was, seeks, encompasses, bord,
Nearest to six: eight, five, four, seven, nine, three, zero, two,
Nearest to seven: nine, eight, five, six, four, three, zero, two,
Nearest to only: eagerly, primarily, eurozone, adversely, strictly, sole, platonist, mourns,
Nearest to were: are, was, these, have, had, those, seem, tend,
Nearest to two: three, four, five, one, six, seven, eight, zero,
Nearest to for: of, in, during, after, by, djamena, despite, krantz,
Nearest to at: in, near, during, beside, attended, goblin, towards, coquitlam,
Nearest to history: timeline, list, overview, heritage, geography, origins, biography, archaeology,
Average loss at step 952000: 2.735506
Average loss at step 954000: 2.505872
Average loss at step 956000: 2.799420
Average loss at step 958000: 2.820224
Average loss at step 960000: 2.966847
Nearest to this: which, it, another, some, cannibalism, thanksgiving, the, whatever,
Nearest to more: less, fewer, most, greater, better, quite, very, worse,
Nearest to system: systems, scheme, method, network, bundesrat, discriminant, cataloging, schemes,
Nearest to his: her, their, my, its, your, our, the, whose,
Nearest to while: whilst, although, though, when, whereas, after, but, besides,
Nearest to after: before, during, despite, afterwards, when, thereafter, while, without,
Nearest to by: through, with, including, from, levin, during, for, via,
Nearest to has: had, have, is, holds, maintains, offers, seeks, having,
Nearest to six: seven, eight, five, four, nine, three, zero, two,
Nearest to seven: six, five, eight, four, nine, three, zero, two,
Nearest to only: eagerly, sole, mourns, adversely, last, eurozone, either, grotto,
Nearest to were: are, was, have, seem, these, had, those, tend,
Nearest to two: three, four, five, six, seven, eight, one, zero,
Nearest to for: after, despite, of, in, against, by, to, during,
Nearest to at: in, during, near, attended, beside, on, headlined, from,
Nearest to history: list, timeline, tradition, heritage, geography, overview, origins, archaeology,
Average loss at step 962000: 2.991305
Average loss at step 964000: 3.014678
Average loss at step 966000: 2.936339
Average loss at step 968000: 3.036912
Average loss at step 970000: 3.058473
Nearest to this: which, it, another, any, physicalism, some, whatever, criminology,
Nearest to more: less, most, fewer, greater, very, quite, fairly, better,
Nearest to system: systems, scheme, network, method, terminology, ecc, procedure, process,
Nearest to his: her, their, my, your, its, our, whose, the,
Nearest to while: whilst, although, after, besides, when, whereas, though, however,
Nearest to after: before, despite, during, while, thereafter, when, afterwards, following,
Nearest to by: including, through, from, via, during, with, wozniak, without,
Nearest to has: had, have, holds, is, maintains, was, requires, contains,
Nearest to six: four, five, eight, seven, three, nine, zero, two,
Nearest to seven: eight, four, six, five, three, nine, zero, two,
Nearest to only: eagerly, actually, sole, adversely, either, grotto, eurozone, mourns,
Nearest to were: are, was, include, have, had, those, although, these,
Nearest to two: three, five, four, six, seven, one, eight, zero,
Nearest to for: against, despite, of, kamov, interdiction, including, without, involves,
Nearest to at: near, during, in, willamette, webcam, beside, attended, outside,
Nearest to history: list, tradition, heritage, archaeology, timeline, origins, scorpions, geography,
Average loss at step 972000: 3.026430
Average loss at step 974000: 3.065782
Average loss at step 976000: 2.838853
Average loss at step 978000: 2.995491
Average loss at step 980000: 3.043791
Nearest to this: which, it, any, whatever, some, physicalism, the, another,
Nearest to more: less, fewer, most, greater, better, smaller, very, quite,
Nearest to system: systems, scheme, network, method, terminology, district, discriminant, model,
Nearest to his: her, their, my, your, its, our, the, innocuous,
Nearest to while: whilst, although, whereas, though, however, when, besides, after,
Nearest to after: before, during, despite, afterwards, without, when, thereafter, while,
Nearest to by: through, without, wozniak, via, from, with, including, walther,
Nearest to has: had, have, is, holds, maintains, requires, bord, offers,
Nearest to six: five, eight, four, seven, nine, three, two, zero,
Nearest to seven: eight, six, four, five, nine, three, zero, two,
Nearest to only: eagerly, eurozone, adversely, mourns, sole, actually, primarily, strictly,
Nearest to were: are, was, those, these, seem, have, although, tend,
Nearest to two: three, four, five, six, seven, one, eight, zero,
Nearest to for: of, without, prototypical, against, krantz, chrominance, after, kamov,
Nearest to at: near, during, in, attended, unlucky, on, beside, milken,
Nearest to history: archaeology, heritage, origins, list, tradition, timeline, literature, overview,
Average loss at step 982000: 2.935518
Average loss at step 984000: 3.043531
Average loss at step 986000: 3.052446
Average loss at step 988000: 3.084532
Average loss at step 990000: 3.010979
Nearest to this: which, it, another, some, any, whatever, physicalism, qibla,
Nearest to more: less, fewer, greater, most, quite, better, worse, smaller,
Nearest to system: systems, scheme, network, platform, method, dos, subsystem, gui,
Nearest to his: her, their, my, your, its, our, whose, innocuous,
Nearest to while: whilst, although, whereas, though, however, moreover, besides, after,
Nearest to after: before, during, despite, without, when, while, afterwards, thereafter,
Nearest to by: through, wozniak, via, including, without, from, for, besides,
Nearest to has: had, have, is, was, holds, requires, offers, maintains,
Nearest to six: four, eight, seven, five, nine, three, zero, two,
Nearest to seven: eight, six, four, nine, five, three, zero, one,
Nearest to only: eagerly, adversely, eurozone, courtly, specifically, necessarily, primarily, mourns,
Nearest to were: are, was, those, these, have, tend, seem, include,
Nearest to two: three, four, five, six, one, seven, eight, zero,
Nearest to for: after, by, including, despite, without, against, of, technician,
Nearest to at: near, during, within, in, on, chronicling, despite, azteca,
Nearest to history: list, timeline, overview, heritage, archaeology, course, origins, geography,
Average loss at step 992000: 2.892751
Average loss at step 994000: 2.770478
Average loss at step 996000: 2.944171
Average loss at step 998000: 3.002622
Average loss at step 1000000: 2.845249
Nearest to this: which, it, adventists, another, the, some, cannibalism, vdp,
Nearest to more: less, fewer, most, greater, quite, better, stronger, bigger,
Nearest to system: systems, network, scheme, method, terminology, platform, schemes, bundesrat,
Nearest to his: her, their, your, my, its, our, the, colleagues,
Nearest to while: whilst, although, whereas, though, after, when, moreover, despite,
Nearest to after: before, during, despite, while, when, without, afterwards, thereafter,
Nearest to by: via, through, during, with, from, wozniak, when, numan,
Nearest to has: had, have, is, maintains, was, having, holds, contains,
Nearest to six: four, eight, seven, five, nine, three, zero, two,
Nearest to seven: five, eight, four, nine, six, three, zero, two,
Nearest to only: adversely, eagerly, actually, eurozone, necessarily, specifically, sole, either,
Nearest to were: are, was, those, have, these, tend, include, seem,
Nearest to two: three, four, five, six, eight, one, seven, nine,
Nearest to for: after, against, despite, of, if, wwf, during, underscore,
Nearest to at: in, during, near, within, on, ophile, for, lombardo,
Nearest to history: timeline, list, geography, overview, archaeology, tradition, career, heritage,
Let us save the embedding for later use.
In [13]:
#### DUMP
#import pickle
#f = open('myembedding.pkl','wb')
#pickle.dump([final_embeddings,dictionary,reverse_dictionary],f)
#f.close()
In [14]:
num_points = 400
tsne = TSNE(perplexity=30, n_components=2, init='pca', n_iter=5000)
two_d_embeddings = tsne.fit_transform(final_embeddings[1:num_points+1, :])
In [15]:
def plot(embeddings, labels):
assert embeddings.shape[0] >= len(labels), 'More labels than embeddings'
pylab.figure(figsize=(15,15)) # in inches
for i, label in enumerate(labels):
x, y = embeddings[i,:]
pylab.scatter(x, y)
pylab.annotate(label, xy=(x, y), xytext=(5, 2), textcoords='offset points',
ha='right', va='bottom')
pylab.show()
words = [reverse_dictionary[i] for i in range(1, num_points+1)]
plot(two_d_embeddings, words)
In the former code we have the dictionary that converts from the word to an index, and the reverse_dictionary that given an index returns the corresponding word.
In [29]:
import pickle
f = open('./dataset/myembedding.pkl','rb')
fe,dic,rdic=pickle.load(f)
f.close()
In [30]:
dic['woman']
Out[30]:
1014
In [31]:
rdic[42]
Out[31]:
'but'
The embedding tries to put together words with similar meaning. A good embedding allows to semantically operate. Let us check some simple semantical operations:
In [32]:
result = (fe[dic['two'],:]+ fe[dic['one'],:])
In [33]:
from scipy.spatial import distance
candidates=np.argsort(distance.cdist(fe,result[np.newaxis,:],metric="seuclidean"),axis=0)
for i in xrange(5):
idx=candidates[i][0]
print(rdic[idx])
two
one
three
four
five
We can also define word analogies: football is to ? as foot is to hand
In [40]:
result = (fe[dic['football'],:] - fe[dic['foot'],:] + fe[dic['hand'],:])
from scipy.spatial import distance
candidates=np.argsort(distance.cdist(fe,result[np.newaxis,:],metric="seuclidean"),axis=0)
for i in xrange(5):
idx=candidates[i][0]
print(rdic[idx])
football
basketball
baseball
sports
hand
In [43]:
result = (fe[dic['madrid'],:] - fe[dic['spain'],:] + fe[dic['germany'],:])
from scipy.spatial import distance
candidates=np.argsort(distance.cdist(fe,result[np.newaxis,:],metric="seuclidean"),axis=0)
for i in xrange(5):
idx=candidates[i][0]
print(rdic[idx])
madrid
germany
berlin
kathryn
rump
In [44]:
result = (fe[dic['barcelona'],:] - fe[dic['spain'],:] + fe[dic['germany'],:])
from scipy.spatial import distance
candidates=np.argsort(distance.cdist(fe,result[np.newaxis,:],metric="seuclidean"),axis=0)
for i in xrange(5):
idx=candidates[i][0]
print(rdic[idx])
barcelona
germany
munich
boziz
liverpool
Let us used a pretrained embedding. We will use a simple embedding detailed in Improving Word Representations via Global Context and Multiple Word Prototypes
In [46]:
import pandas as pd
df = pd.read_table("./dataset/wordVectors.txt",delimiter=" ",header=None)
embedding=df.values[:,:-1]
In [48]:
f = open("./dataset/vocab.txt",'r')
dictionary=dict()
for word in f.readlines():
dictionary[word] = len(dictionary)
reverse_dictionary = dict(zip(dictionary.values(), dictionary.keys()))
In [51]:
result = embedding[dictionary['king\n'],:]-embedding[dictionary['man\n'],:]+embedding[dictionary['girl\n'],:]
import numpy as np
from scipy.spatial import distance
candidates=np.argsort(distance.cdist(embedding,result[np.newaxis,:],metric="seuclidean"),axis=0)
for i in xrange(0,5):
idx=candidates[i][0]
print(reverse_dictionary[idx])
prince
king
princess
queen
bride
In [ ]:
Content source: jvitria/DeepLearningBBVA2016
Similar notebooks: